diff --git a/composeApp/src/commonMain/composeResources/values/strings.xml b/composeApp/src/commonMain/composeResources/values/strings.xml index f6e0cc25..161e370e 100644 --- a/composeApp/src/commonMain/composeResources/values/strings.xml +++ b/composeApp/src/commonMain/composeResources/values/strings.xml @@ -495,4 +495,10 @@ Choose chapter files That file isn't text. A chapter is markdown or plain text. That file is too large to be a chapter. + This group has no shared key, so it cannot sign a chapter into existence. Run a shared key ceremony first. + %1$d chunks is more than the group can sign in one go. Split the chapter so that no part of it is over %2$d paragraphs. + This chapter is too long to propose. Asking the group to sign it and its %1$d chunks would send %2$s, and a group message carries %3$s. Split it in two. + %1$d words · %2$d characters · %3$s · %4$d of %5$d chunks + %1$s of %2$s + %1$s KiB diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/FrostSigningManager.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/FrostSigningManager.kt index 5dfa9ce4..59798fd1 100644 --- a/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/FrostSigningManager.kt +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/FrostSigningManager.kt @@ -146,6 +146,29 @@ object FrostSigningManager { */ const val MAX_BATCH_SIZE: Int = 64 + /** + * The most JSON a proposal may carry. + * + * [MAX_BATCH_SIZE] bounds how many events a batch holds; this bounds how + * large they are, and the two are not the same limit. Sixty-four dialects + * are a few kilobytes and sixty-four chunks of a chapter are eighty, so a + * count cap alone lets a perfectly legal batch exceed what the transport + * carries -- and the proposal is the largest message in this protocol by an + * order of magnitude: a signer's nonce payload is `k * 133` bytes of hex and + * their partial `k * 65`, where the proposal carries `k` whole events. + * + * Deliberately the same number as `ChronicleEvent.MAX_PAGE_BYTES`, because + * it is the same question asked twice -- what one kind:445 clears once the + * MLS framing and the NIP-44 expansion outside it are paid for. The two are + * not shared as one constant because neither module should depend on the + * other for it, but they do have to be re-sized together, and whichever is + * raised first should say so here. + * + * Before this existed the failure was silent: the proposal went out, the + * group event was over size, and the session stalled with nothing to blame. + */ + const val MAX_PROPOSAL_BYTES: Int = 64 * 1024 + /** * Opens a signing session, making this device the coordinator. * @@ -270,6 +293,16 @@ object FrostSigningManager { "A signing session will sign at most $MAX_BATCH_SIZE events, not ${unsignedEvents.size}" } + // Encoded once, here, so the size check is the real encoding rather than + // an estimate of it -- and so the broadcast below sends the bytes that + // were measured. + val proposal = FrostSigningEvents.encodeProposal(unsignedEvents) + val proposalBytes = proposal.encodeToByteArray().size + + require(proposalBytes <= MAX_PROPOSAL_BYTES) { + "A signing proposal carries at most $MAX_PROPOSAL_BYTES bytes, not $proposalBytes" + } + val sessionId = RandomInstance.bytes(32).toHex() val session = FrostSigningSession( @@ -299,7 +332,7 @@ object FrostSigningManager { localChatRoom = localChatRoom, session = session, kind = FrostSigningEvents.PROPOSAL, - content = FrostSigningEvents.encodeProposal(unsignedEvents), + content = proposal, includeKey = true ) @@ -406,6 +439,18 @@ object FrostSigningManager { return null } + // Before parsing, not after. This and the count check below are the two + // places a remote party decides how much work this device does, and the + // cheaper of the two refusals should come first. + val proposalBytes = innerEvent.content.encodeToByteArray().size + if (proposalBytes > MAX_PROPOSAL_BYTES) { + logger.w( + "Signing proposal $sessionId carries $proposalBytes bytes, more than the " + + "$MAX_PROPOSAL_BYTES this device will accept; dropping" + ) + return null + } + val proposed = FrostSigningEvents.decodeProposal(innerEvent.content) if (proposed == null) { logger.w("Signing proposal $sessionId does not carry any events; dropping") diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/nip30303/ChapterProposal.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/nip30303/ChapterProposal.kt new file mode 100644 index 00000000..0fb718c5 --- /dev/null +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/nip30303/ChapterProposal.kt @@ -0,0 +1,94 @@ +package press.mantra.compose.nostr.nip30303 + +import press.mantra.compose.managers.FrostSigningManager +import press.mantra.compose.text.Markdown + +/** + * What proposing a chapter will cost on the wire, close enough to say so while + * it is being typed. + * + * `FrostSigningManager.MAX_PROPOSAL_BYTES` is the limit and the manager enforces + * it exactly, by encoding the batch and measuring it. That answer arrives too + * late to be useful: by then the chapter has been built, the chunks cut and the + * session is one line from being opened, and the only thing left to do with the + * number is throw. This is the same question answered cheaply enough to put in + * front of somebody with a cursor still in the text field. + * + * **Why a chapter needs this at all.** A chapter travels with a chunk per + * paragraph, and the chapter carries the whole markdown while the chunks carry + * the same words again between them -- see `ChapterEvent.originalText` for why + * that duplication is wanted. So a proposal is a little over twice the prose, + * plus an envelope per event, and the limit arrives at about 20 KiB of markdown + * at the maximum chunk count. That is inside the range of a real book chapter. + * + * **Why this is not a constant.** A flat `MAX_ORIGINAL_TEXT_BYTES` was the + * obvious shape and is wrong by half. The envelopes are what varies: measured, + * a 63-chunk chapter fits about 20 KiB of markdown and a 10-chunk chapter fits + * about 30. A single number safe for the first refuses a third of the second's + * text for no reason, and a single number sized for the second stalls sessions. + * The count is known as the text is typed, so the limit may as well be. + */ +object ChapterProposal { + + /** + * What one unsigned event costs beside its own text. + * + * **Measured, not derived.** Across chapters from 1 KiB to 24 KiB and from 1 + * to 63 chunks, the per-event cost of an encoded proposal runs between 357 + * and 388 bytes: a 64-character id, a 64-character pubkey, an empty `sig` + * (a proposal carries unsigned events, which is why this is smaller than a + * signed event's ~370-byte floor), the JSON structure, the alt tag, and -- on + * a chunk -- the 64-character chapter id it points at. + * + * Rounded up to 448 rather than to the measured maximum, and that direction + * is the deliberate one. Over-estimating costs an author a slightly shorter + * chapter; under-estimating costs the group a session that stalls with + * nothing to blame. `ChapterProposalSizeTest` asserts the estimate stays at + * or above the real encoding, so a tag added to either kind fails a test + * rather than a quorum. + */ + const val EVENT_OVERHEAD_BYTES: Int = 448 + + /** The limit, which is the manager's and not this file's to choose. */ + const val MAX_PROPOSAL_BYTES: Int = FrostSigningManager.MAX_PROPOSAL_BYTES + + /** + * What a proposal for this chapter would weigh. + * + * Takes the chunks rather than re-splitting, because every caller has just + * counted them for the other cap and because summing their text is exact + * where doubling the chapter's would not be -- the blank lines between + * paragraphs are in the chapter's text and in no chunk's. + */ + fun proposalBytes(originalText: String, chunks: List): Int = + originalText.utf8Size() + + chunks.sumOf { it.utf8Size() } + + (chunks.size + 1) * EVENT_OVERHEAD_BYTES + + fun proposalBytes(originalText: String): Int = + proposalBytes(originalText, Markdown.splitParagraphs(originalText)) + + fun fits(originalText: String, chunks: List): Boolean = + proposalBytes(originalText, chunks) <= MAX_PROPOSAL_BYTES + + /** + * The most markdown a chapter of this many chunks may carry. + * + * For the form, which has to say what the limit *is* and not only that it + * has been passed. Inverts [proposalBytes]: the envelopes come off the top + * and the prose is counted twice, so what is left is halved. + */ + fun maxOriginalTextBytes(chunkCount: Int): Int = + ((MAX_PROPOSAL_BYTES - (chunkCount + 1) * EVENT_OVERHEAD_BYTES) / 2) + .coerceAtLeast(0) +} + +/** + * The bytes a string takes as UTF-8, which is what the wire counts. + * + * Not `length`, which counts UTF-16 code units: the languages this app exists + * for are full of characters that are one unit and three bytes, so measuring a + * Setswana or Amharic chapter by its `length` would under-count it by half and + * the cap would stop binding exactly where it matters most. + */ +private fun String.utf8Size(): Int = encodeToByteArray().size diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/text/Kibibytes.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/text/Kibibytes.kt new file mode 100644 index 00000000..fdbea5a9 --- /dev/null +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/text/Kibibytes.kt @@ -0,0 +1,24 @@ +package press.mantra.compose.text + +/** + * A byte count as the number part of "19.9 KiB". + * + * The unit itself is a string resource, so this returns only the figure. + * + * One decimal place, by integer arithmetic rather than through a float format. + * Two reasons, and the first is the one that matters: the figures this renders + * sit next to each other -- what a chapter weighs, and what the group can carry + * -- and whole kibibytes would print a 20,473-byte chapter and its 21,226-byte + * limit as the same "19 KiB", which reads as a form refusing something it says + * is within the limit. The second is that there is no common-code decimal + * formatter to reach for, and the three platforms' own would each punctuate it + * differently. + * + * Truncating rather than rounding, for the same reason: a figure shown as under + * the limit has to be under it. + */ +internal fun kibibytes(bytes: Int): String { + val whole = bytes / 1024 + val tenths = (bytes % 1024) * 10 / 1024 + return "$whole.$tenths" +} diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/composable/AddChapterScreen.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/composable/AddChapterScreen.kt index 9465b790..98ee337a 100644 --- a/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/composable/AddChapterScreen.kt +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/composable/AddChapterScreen.kt @@ -42,9 +42,11 @@ import press.mantra.compose.repository.ChatRepository import press.mantra.compose.repository.FrostSigningRepository import press.mantra.compose.repository.MantraRepository import press.mantra.compose.files.rememberFilePicker +import press.mantra.compose.nostr.nip30303.ChapterProposal import press.mantra.compose.text.ChapterNaming import press.mantra.compose.text.ChapterText import press.mantra.compose.text.Markdown +import press.mantra.compose.text.kibibytes import press.mantra.compose.ui.composable.navigation.routes.FrostSigningRoute import press.mantra.compose.ui.composable.navigation.routes.ImplementationPendingRoute import press.mantra.compose.ui.composable.navigation.routes.Route @@ -64,6 +66,12 @@ import mantra.composeapp.generated.resources.propose_chapter import mantra.composeapp.generated.resources.this_artifact_has_no_version_for_a_chapter import mantra.composeapp.generated.resources.add_chapter_to import mantra.composeapp.generated.resources.choose_chapter_files +import mantra.composeapp.generated.resources.d_chunks_is_more_than_the_group_can_sign +import mantra.composeapp.generated.resources.s_kib +import mantra.composeapp.generated.resources.s_of_s +import mantra.composeapp.generated.resources.this_chapter_is_too_long_to_propose +import mantra.composeapp.generated.resources.this_group_has_no_shared_key_for_a_chapter +import mantra.composeapp.generated.resources.words_characters_size_chunks import mantra.composeapp.generated.resources.load_from_file import mantra.composeapp.generated.resources.that_file_is_too_large import mantra.composeapp.generated.resources.that_file_isn_t_text @@ -160,7 +168,8 @@ fun AddChapterScreen( val originalText = originalTextFieldState.text.toString() val wordCount = Markdown.wordCount(originalText) val characterCount = Markdown.characterCount(originalText) - val paragraphCount = Markdown.splitParagraphs(originalText).size + val chunks = Markdown.splitParagraphs(originalText) + val paragraphCount = chunks.size // A chapter hangs off a version, and the group has to be able to // sign; without both there is nothing this screen can propose. @@ -172,10 +181,19 @@ fun AddChapterScreen( // is still a cursor in the text rather than after a failed propose. val tooManyChunks = paragraphCount > AddChapterViewModel.MAX_CHUNKS_PER_CHAPTER + // The other cap, and the one that had been enforced nowhere: a + // proposal carries the chapter whole and every chunk cut from + // it, so it weighs a little over twice the prose plus an + // envelope per event. Past what a group event holds, the session + // used to go out and stall. + val proposalBytes = ChapterProposal.proposalBytes(originalText, chunks) + val tooLargeToPropose = !tooManyChunks && proposalBytes > ChapterProposal.MAX_PROPOSAL_BYTES + val canProposeChapter = artifactVersion != null && addChapterUIState.canSign && paragraphCount > 0 && - !tooManyChunks + !tooManyChunks && + !tooLargeToPropose // imePadding: this screen has a text field, and without it the software // keyboard covers whatever is being typed into. @@ -236,8 +254,7 @@ fun AddChapterScreen( ) { if (!addChapterUIState.canSign) { Text( - text = "This group has no shared key, so it cannot sign a " + - "chapter into existence. Run a shared key ceremony first.", + text = stringResource(Res.string.this_group_has_no_shared_key_for_a_chapter), style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.error ) @@ -249,9 +266,25 @@ fun AddChapterScreen( ) } else if (tooManyChunks) { Text( - text = "$paragraphCount chunks is more than the group can sign in " + - "one go. Split the chapter so that no part of it is over " + - "${AddChapterViewModel.MAX_CHUNKS_PER_CHAPTER} paragraphs.", + text = stringResource( + Res.string.d_chunks_is_more_than_the_group_can_sign, + paragraphCount, + AddChapterViewModel.MAX_CHUNKS_PER_CHAPTER, + ), + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.error + ) + } else if (tooLargeToPropose) { + Text( + text = stringResource( + Res.string.this_chapter_is_too_long_to_propose, + paragraphCount, + stringResource(Res.string.s_kib, kibibytes(proposalBytes)), + stringResource( + Res.string.s_kib, + kibibytes(ChapterProposal.MAX_PROPOSAL_BYTES), + ), + ), style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.error ) @@ -287,14 +320,28 @@ fun AddChapterScreen( ) Text( - // The chunk count is what the group is asked to sign - // alongside the chapter, so it is a count of the work - // being proposed rather than a curiosity about the text. - text = "$wordCount words · $characterCount characters · " + - "$paragraphCount ${if (paragraphCount == 1) "chunk" else "chunks"} " + - "of ${AddChapterViewModel.MAX_CHUNKS_PER_CHAPTER}", + // Both caps, as the text is typed. The chunk count and + // the proposal's size are what the group is asked to + // carry, so they are a measure of the work being + // proposed rather than a curiosity about the text -- + // and either of them can be the one that binds. + text = stringResource( + Res.string.words_characters_size_chunks, + wordCount, + characterCount, + stringResource( + Res.string.s_of_s, + stringResource(Res.string.s_kib, kibibytes(proposalBytes)), + stringResource( + Res.string.s_kib, + kibibytes(ChapterProposal.MAX_PROPOSAL_BYTES), + ), + ), + paragraphCount, + AddChapterViewModel.MAX_CHUNKS_PER_CHAPTER, + ), style = MaterialTheme.typography.labelMedium, - color = if (tooManyChunks) { + color = if (tooManyChunks || tooLargeToPropose) { MaterialTheme.colorScheme.error } else { MaterialTheme.typography.labelMedium.color diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/AddChapterViewModel.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/AddChapterViewModel.kt index bbe3e9d7..a7dc3e4a 100644 --- a/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/AddChapterViewModel.kt +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/AddChapterViewModel.kt @@ -21,6 +21,7 @@ import press.mantra.compose.database.model.intermdiate.LocalChatRoom import press.mantra.compose.managers.FrostSigningManager import press.mantra.compose.managers.TranslationScaffold import press.mantra.compose.nostr.nip30303.ChapterEvent +import press.mantra.compose.nostr.nip30303.ChapterProposal import press.mantra.compose.nostr.nip30303.ChunkEvent import press.mantra.compose.repository.ChatRepository import press.mantra.compose.repository.FrostSigningRepository @@ -102,10 +103,19 @@ class AddChapterViewModel( val originalText = originalTextField.text.toString() val paragraphs = Markdown.splitParagraphs(originalText) - // The cap is the group's, not this screen's, and proposing past it - // throws rather than failing softly. The form says so as it is typed; - // this is the check that has to hold when it does not. - if (name.isBlank() || paragraphs.isEmpty() || paragraphs.size > MAX_CHUNKS_PER_CHAPTER) { + // Both caps are the group's, not this screen's, and proposing past + // either throws rather than failing softly. The form says so as it is + // typed; these are the checks that have to hold when it does not. + // + // Two caps rather than one because they bound different things: how many + // events a session signs, and how many bytes the proposal carrying them + // weighs. A chapter of sixty-three one-word paragraphs passes the second + // and fails the first; one of three very long ones does the reverse. + if (name.isBlank() || + paragraphs.isEmpty() || + paragraphs.size > MAX_CHUNKS_PER_CHAPTER || + !ChapterProposal.fits(originalText, paragraphs) + ) { onFailure.invoke() return } diff --git a/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/nip30303/ChapterProposalSizeTest.kt b/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/nip30303/ChapterProposalSizeTest.kt new file mode 100644 index 00000000..dc9c477e --- /dev/null +++ b/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/nip30303/ChapterProposalSizeTest.kt @@ -0,0 +1,197 @@ +package press.mantra.compose.nostr.nip30303 + +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.nip01Core.crypto.EventHasher +import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue +import press.mantra.compose.managers.FrostSigningManager +import press.mantra.compose.nostr.frost.FrostSigningEvents +import press.mantra.compose.text.Markdown + +/** + * [ChapterProposal] estimates what the manager measures, and the only property + * worth asserting is that it never estimates low. + * + * The form reads the estimate as the text is typed; `proposeSigningBatch` + * encodes the batch and measures it for real. If the estimate were ever under + * the truth, the form would offer a chapter that the manager then refuses with + * an `IllegalArgumentException` -- or, before that check existed, that went out + * and stalled the session with nothing to blame. + * + * So these tests build the same events the manager would and compare. A tag + * added to either kind fails here rather than in a room. + */ +class ChapterProposalSizeTest { + private val groupKey = "a".repeat(64) + private val artifactVersionId = "b".repeat(64) + + private fun unsigned(template: EventTemplate<*>) = Event( + id = EventHasher.hashId( + pubKey = groupKey, + createdAt = template.createdAt, + kind = template.kind, + tags = template.tags, + content = template.content, + ), + pubKey = groupKey, + createdAt = template.createdAt, + kind = template.kind, + tags = template.tags, + content = template.content, + sig = "", + ) + + /** The batch as `proposeSigningBatch` would build it: the chapter, then its chunks. */ + private fun batchFor(originalText: String, name: String = "Chapter 1"): List { + val chapter = unsigned( + ChapterEvent.build( + artifactVersionId = artifactVersionId, + name = name, + originalText = originalText, + index = 0, + wordCount = Markdown.wordCount(originalText), + characterCount = Markdown.characterCount(originalText), + createdAt = 1_700_000_000, + ) + ) + return listOf(chapter) + ChunkEvent.splitOf(chapter).map(::unsigned) + } + + private fun encodedBytes(originalText: String): Int = + FrostSigningEvents.encodeProposal(batchFor(originalText)).encodeToByteArray().size + + /** Markdown of roughly [bytes] bytes, in [paragraphs] paragraphs. */ + private fun markdownOf(bytes: Int, paragraphs: Int): String { + val perParagraph = (bytes / paragraphs).coerceAtLeast(4) + return (0 until paragraphs).joinToString("\n\n") { "w ".repeat(perParagraph / 2).trim() } + } + + @Test + fun `the estimate is never under the real encoding`() { + // The grid the overhead constant was measured over. Both ends matter: at + // one chunk the text is counted twice exactly and the per-event cost is + // at its highest, which is where an estimate is most likely to fall + // short; at sixty-three the envelopes dominate. + val shapes = listOf(1, 2, 5, 10, 20, 40, 63) + val sizes = listOf(1, 4, 8, 16, 20, 24) + + for (paragraphs in shapes) { + for (kib in sizes) { + val text = markdownOf(kib * 1024, paragraphs) + val chunks = Markdown.splitParagraphs(text) + val estimated = ChapterProposal.proposalBytes(text, chunks) + val actual = encodedBytes(text) + + assertTrue( + estimated >= actual, + "estimate $estimated under the real $actual for $paragraphs " + + "paragraphs of ${kib} KiB", + ) + } + } + } + + @Test + fun `the estimate is close enough to be worth showing`() { + // Not merely conservative -- useful. An estimate twice the truth would + // pass the test above and refuse half of every legal chapter. + val text = markdownOf(16 * 1024, 40) + val chunks = Markdown.splitParagraphs(text) + + val estimated = ChapterProposal.proposalBytes(text, chunks) + val actual = encodedBytes(text) + + assertTrue( + estimated <= actual + actual / 10, + "estimate $estimated is more than a tenth over the real $actual", + ) + } + + @Test + fun `a chapter the estimate accepts really does encode under the cap`() { + // The property the form's offer rests on, stated as a round trip rather + // than as arithmetic: find the longest text the estimate allows at the + // worst chunk count, and check the manager would carry it. + val chunkCount = AddChapterMaxChunks + val allowance = ChapterProposal.maxOriginalTextBytes(chunkCount) + val text = markdownOf(allowance, chunkCount) + val chunks = Markdown.splitParagraphs(text) + + assertEquals(chunkCount, chunks.size) + assertTrue(ChapterProposal.fits(text, chunks)) + assertTrue( + encodedBytes(text) <= FrostSigningManager.MAX_PROPOSAL_BYTES, + "a chapter the form accepts encoded to ${encodedBytes(text)} bytes, over the cap", + ) + } + + @Test + fun `the cap binds inside the range of a real chapter`() { + // Why any of this exists. Before the check, a chapter of this size was + // offered by the form, proposed, and stalled. 28 KiB over forty + // paragraphs is a long chapter of a real book, not a pathological one -- + // and measured, the fit runs out at about 24 KiB for that shape. + val text = markdownOf(28 * 1024, 40) + val chunks = Markdown.splitParagraphs(text) + + assertTrue( + encodedBytes(text) > FrostSigningManager.MAX_PROPOSAL_BYTES, + "expected ${encodedBytes(text)} bytes to exceed the cap", + ) + assertTrue(!ChapterProposal.fits(text, chunks)) + } + + @Test + fun `where the fit runs out, for a chapter-shaped chapter`() { + // Pinned so the number is on the record rather than in a comment, and so + // that a change to either event's tags shows up as a moved boundary + // instead of as a quorum that never arrives. Forty paragraphs is the + // shape; the figure is the largest whole KiB of markdown that encodes + // under the cap. + val fitting = (1..64).last { kib -> + encodedBytes(markdownOf(kib * 1024, 40)) <= FrostSigningManager.MAX_PROPOSAL_BYTES + } + + assertEquals(24, fitting) + } + + @Test + fun `an ordinary chapter is nowhere near either cap`() { + // The other half of the claim: this is a limit on very long chapters, + // not a tax on normal ones. Twelve paragraphs of a few hundred words. + val text = markdownOf(8 * 1024, 12) + val chunks = Markdown.splitParagraphs(text) + + assertTrue(ChapterProposal.fits(text, chunks)) + assertTrue(encodedBytes(text) < FrostSigningManager.MAX_PROPOSAL_BYTES / 2) + } + + @Test + fun `the text is measured in bytes, not in characters`() { + // The languages this app exists for are full of characters that are one + // UTF-16 unit and three bytes. Measuring by length would under-count a + // Setswana or Amharic chapter and the cap would stop binding exactly + // where it matters most. + val ascii = "w ".repeat(512).trim() + val amharic = "ሰ ".repeat(512).trim() + + assertEquals(ascii.length, amharic.length) + assertTrue( + ChapterProposal.proposalBytes(amharic, listOf(amharic)) > + ChapterProposal.proposalBytes(ascii, listOf(ascii)), + ) + } + + /** + * Named here rather than imported from the view model, so this file depends + * on the protocol and not on a screen. Asserted equal below. + */ + private val AddChapterMaxChunks: Int get() = FrostSigningManager.MAX_BATCH_SIZE - 1 + + @Test + fun `the chunk cap is one less than the batch, because the chapter takes a place`() { + assertEquals(FrostSigningManager.MAX_BATCH_SIZE - 1, AddChapterMaxChunks) + } +} diff --git a/docs/chapter-uploads.md b/docs/chapter-uploads.md index 0ad6c375..d5aefa5a 100644 --- a/docs/chapter-uploads.md +++ b/docs/chapter-uploads.md @@ -289,36 +289,50 @@ title; `03 - The Wreck.md` is. **Half a day. A fix to the path that exists, and a prerequisite for Phase 4.** -A chapter whose markdown is over roughly 28 KiB cannot be proposed today and +A chapter whose markdown is over roughly 24 KiB cannot be proposed today and nothing says so. Make the limit explicit and check it where the paragraph cap is already checked. -```kotlin -// ChapterEvent companion, beside build() -/** - * The most markdown a chapter may carry. - * - * A chapter is proposed with a chunk per paragraph, so the proposal carries its - * text roughly twice -- whole in the chapter's `original_text` tag, and again - * split across the chunks -- before any framing. The group event that carries - * the proposal is sized by ChronicleEvent.MAX_PAGE_BYTES; half of it, less room - * for k event envelopes, is what is left for prose. - * - * Measure it against a real proposal rather than trusting this arithmetic, the - * way MAX_PAGE_BYTES itself says to. - */ -const val MAX_ORIGINAL_TEXT_BYTES = 24 * 1024 -``` +**Built differently, in three ways, and the measurement is why.** Encoding real +proposals gave a per-event overhead of 357–388 bytes across chapters from 1 KiB +to 24 KiB and from 1 to 63 chunks — the plan's ~370 estimate, arrived at for the +wrong reason: a proposal carries *unsigned* events, so there is no 128-character +signature in it, and the figure is instead an id, a pubkey, an empty `sig`, the +JSON structure and, on a chunk, the 64-character chapter id it points at. -24 KiB rather than 28: the envelopes are not free (an id, a pubkey and a -signature are ~370 bytes before an event says anything, times `k`), and a cap -that is comfortably under the wire limit fails in the form instead of on the -wire. It should be arrived at by measuring an encoded proposal for a 63-chunk -chapter, not by this paragraph. +What that measurement changed: -**And the number is provisional until Phase 8 lands.** Block splitting produces -more chunks than blank-line splitting, so the same prose carries more envelopes; -whichever of the two phases goes second re-runs the measurement. +**One. A flat `MAX_ORIGINAL_TEXT_BYTES` is wrong by half.** Measured, a +63-chunk chapter fits about 20 KiB of markdown and a 10-chunk chapter fits about +30, because the envelopes are what varies. A single constant safe for the first +refuses a third of the second's text for nothing; one sized for the second +stalls sessions. The chunk count is known as the text is typed, so the limit may +as well be — `ChapterProposal.proposalBytes(text, chunks)` and +`maxOriginalTextBytes(chunkCount)` instead of a constant. + +**Two. The limit belongs to the manager, not to the chapter.** +`FrostSigningManager.MAX_PROPOSAL_BYTES = 64 * 1024`, beside `MAX_BATCH_SIZE`, +enforced in `proposeSigningBatch` against the real encoding and *independently* +in `acceptProposal` — before parsing, since that is the cheaper refusal. The +count cap already had exactly this two-sided treatment for exactly this reason: +the proposal is where a remote party decides how much work everybody else does, +and until now its *size* was unbounded. Both propose forms funnel into the +`lead`/`dependents` one, so a single check covers them. + +**Three. `ChapterProposal` estimates what the manager measures.** The exact +answer needs the chapter built and the chunks cut, which is one line from +opening a session — too late to tell somebody with a cursor in a text field. +So: a cheap estimate for the form at 448 bytes per event, rounded up from the +measured 388, and a test asserting the estimate never falls under the real +encoding. Over-estimating costs an author a slightly shorter chapter; +under-estimating costs the group a stalled session, so the rounding goes that +way on purpose. + +**The number is still provisional until Phase 8 lands.** Block splitting +produces more chunks than blank-line splitting, so the same prose carries more +envelopes; `ChapterProposalSizeTest` pins where the fit runs out (24 KiB over +forty paragraphs), so that phase will move a pinned figure rather than quietly +narrow the limit. Where it goes: @@ -326,8 +340,11 @@ Where it goes: `paragraphs.size > MAX_CHUNKS_PER_CHAPTER` guard, same early `onFailure`. - `AddChapterScreen` — the counter line already reads `"$wordCount words · $characterCount characters · $paragraphCount chunks of N"`; - it gains the byte figure against the cap and turns `error`-coloured on the - same rule the chunk count does. + it gains the proposal's size against the cap and turns `error`-coloured on the + same rule the chunk count does. The message compares like with like — what + proposing would send, against what a group message carries — rather than a + character count against a byte budget, which was the first draft of it and was + comparing UTF-16 units to bytes. - The two existing hardcoded English literals on that screen (the no-shared-key message and the too-many-chunks message) move into `composeApp/src/commonMain/composeResources/values/strings.xml` while the file @@ -335,13 +352,15 @@ Where it goes: title-case strings with literals reported; a third one added by this work would be the wrong direction. -**Test:** extend `ChapterBatchProposalJvmTest`, which already asserts the -paragraph cap's boundary (`MAX_BATCH_SIZE - 1` proposes, `MAX_BATCH_SIZE` -throws). Add the byte boundary the same way: one chapter at the cap proposes, -one a byte over does not. And one that measures the *encoded proposal* for a -cap-sized, 63-chunk chapter and asserts it is under `MAX_PAGE_BYTES` — that is -the assertion that makes the constant honest, and the only one that will catch -it drifting when a tag is added to either kind. +**Tests:** a `ChapterProposalSizeTest` in `commonTest` — pure, since the +estimate and the encoding are both pure — over the grid the overhead was +measured on, asserting the estimate never falls under the real encoding and +stays within a tenth of it. Plus the round trip the form's offer rests on: the +longest text the estimate allows at the worst chunk count really does encode +under the cap. Plus the boundary pinned as a number, so a tag added to either +kind moves a figure in a test rather than producing a quorum that never +arrives. And the byte-vs-character one, in Amharic, because an ASCII-only test +suite would never have caught the UTF-16 slip above. ---