test: prove the catch-up row by row, and say what an old build makes of a page

Phases 8 and 9 of docs/member-archive.md. The tests ran in the phases where the
code they cover first existed -- the way the batch-signing note's did -- so this
is what was missing from them, plus the rollout note, plus the plan marked built.

**Compared row by row, not by count.** The end-to-end test asserted the two
databases held the same *number* of artifacts, chapters and chunks. That is not
the claim: two databases can hold the same counts and disagree about every row,
and a rebuild that lost the group's signature -- or re-authored a row as whoever
sent it -- would pass a count and fail the only thing an archive is for. It now
compares `(id, author, signature)` per row across every archived kind, and then
asserts each one is authored by the room and carries a signature.

The artifact version is the one exception, and it has to be: nobody signs it, it
is derived from the signed artifact on arrival. Which is exactly why it is not
archived, and why a chapter's foreign key survives without it.

**An old build does not ignore an archive page, it renders it.** Phase 9's first
draft said an old build "files it as unsupported, exactly as it does today for
anything it does not know" -- true, and it reads better than it lives. An
unsupported row's content is `event.toJson()` and it renders as an ordinary chat
bubble, so every member on an old build sees each archive page as a raw-JSON
bubble of up to `MAX_PAGE_BYTES`, once per page.

Nothing breaks and nothing is lost, but a group mid-upgrade gets a genuinely
unpleasant transcript, and that is worth knowing before the first archive goes
out. So the rollout rule is stated rather than implied: the receiving half ships
safely on its own -- phases 1-4 send nothing -- and no member starts sending
until every member understands kind 30327. The mitigation if that ever proves
unacceptable is the one the appendix rejects for other reasons, and it is named
there so the trade can be weighed rather than rediscovered.

**The plan is marked built**, with a table of the five places the implementation
chose differently from the plan and why: nine archivable kinds became six, a
count cap that could never fire, queueing moved a phase later, a re-read that was
never needed, and the rollout note above. Phase 8 also records the three tests
that were not in the first draft, each written because something passed for the
wrong reason -- a cap that could not fire, an out-of-order test on an archive
that was never out of order, and a sweep whose "still missing" count included
failures a later pass had already fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/ArchiveApplyJvmTest.kt

@@ -368,6 +368,42 @@ class ArchiveApplyJvmTest {
         )
     }
 
+    /**
+     * Every archived row as (id, author, signature), sorted.
+     *
+     * Counting is not the claim. Two databases can hold the same number of
+     * artifacts and disagree about every one of them, and a rebuild that lost the
+     * group's signature -- or re-authored a row as whoever sent it -- would pass a
+     * count and fail the only thing the archive is for.
+     */
+    private suspend fun rowFingerprints(db: MantraDatabase): List<Triple<String, String, String>> {
+        val artifacts = db.mantraArtifactDao().getArtifactsByChatRoomId(chatRoomId)
+        val versions = artifacts.flatMap {
+            db.mantraArtifactVersionDao().getArtifactVersionsByArtifactId(it.id)
+        }
+        val chapters = versions.flatMap { db.mantraChapterDao().getChaptersByArtifactVersionId(it.id) }
+        val translationVersions = versions.flatMap {
+            db.mantraTranslationArtifactVersionDao().getTranslationsByArtifactVersionId(it.id)
+        }
+
+        return buildList {
+            db.mantraDialectDao().getDialectsByChatRoomId(chatRoomId)
+                .forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+            artifacts.forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+            versions.forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+            chapters.forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+            chapters.flatMap { db.mantraChunkDao().getChunksByChapterId(it.id) }
+                .forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+            translationVersions.forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+            translationVersions
+                .flatMap {
+                    db.mantraTranslationChapterDao()
+                        .getTranslationChaptersByTranslationArtifactVersionId(it.id)
+                }
+                .forEach { add(Triple(it.id, it.publicKey, it.signature)) }
+        }.sortedBy { it.first }
+    }
+
     @Test
     fun `a member who was never there ends up holding what the group signed`() = runBlocking {
         seedSenderWork()
@@ -386,14 +422,26 @@ class ArchiveApplyJvmTest {
 
         assertEquals(rowCounts(sender), rowCounts(receiver))
 
-        // Not merely the same shape: the same rows, with the group's signature on
+        // Not merely the same shape: the same rows, each with the group's own
-        // them. An archive that produced look-alike rows authored by the sender
+        // signature on it. An archive that produced look-alike rows authored by
-        // would pass a count and fail the only thing that matters.
+        // the member who sent them would pass a count and fail the only thing
-        val theirs = receiver.mantraArtifactDao().getArtifactsByChatRoomId(chatRoomId).single()
+        // this is for.
-        val ours = sender.mantraArtifactDao().getArtifactsByChatRoomId(chatRoomId).single()
+        val fingerprints = rowFingerprints(receiver)
-        assertEquals(ours.id, theirs.id)
+        assertEquals(rowFingerprints(sender), fingerprints)
-        assertEquals(ours.signature, theirs.signature)
+
-        assertEquals(chatRoomId, theirs.publicKey)
+        // And every one of them is the room's own work rather than anybody's.
+        // The artifact version is the exception and has to be: nobody signs it,
+        // it is derived from the signed artifact on arrival, which is exactly why
+        // it is not archived and why a chapter's foreign key survives anyway.
+        val versionIds = receiver.mantraArtifactDao().getArtifactsByChatRoomId(chatRoomId)
+            .flatMap { receiver.mantraArtifactVersionDao().getArtifactVersionsByArtifactId(it.id) }
+            .map { it.id }
+            .toSet()
+
+        fingerprints.filterNot { it.first in versionIds }.forEach { (id, author, signature) ->
+            assertEquals(chatRoomId, author, "row $id is not authored by the room")
+            assertTrue(signature.isNotBlank(), "row $id came across without a signature")
+        }
     }
 
     @Test

docs/member-archive.md

@@ -9,6 +9,24 @@ Read [shared-key-derivation.md](./shared-key-derivation.md) first. The property
 this whole design rests on -- that a room's id *is* the key it signs with -- is
 stated there, and everything cheap about what follows is downstream of it.
 
+**Built**, phases 1-9, one commit each, with two exceptions named in Phase 7. The
+phases are kept as written because they are the reasoning, and the code reads
+better against the argument it came from than against a summary of itself. Where
+the implementation chose differently the section says so, and it did so five
+times worth reading:
+
+| what the plan said | what it turned out to be |
+|---|---|
+| nine archivable kinds | six. Only six ever reach a signing session, and one of the three that do not is the translated text -- see [Phase 3](#what-is-actually-archivable-which-is-less-than-it-looks) |
+| `MAX_PAGE_EVENTS = 256` | 128. At 256 the byte cap always binds first and the count cap can never fire |
+| "assemble, order, pack and queue" | assemble only; queueing moved to Phase 5, next to the thing that decides when |
+| "re-read the room between the invite and the assembly" | unnecessary; that rule is about the MLS snapshot a commit is built on |
+| an old build "files it as unsupported" and nothing breaks | true, and it renders as a raw-JSON chat bubble per page -- see [Phase 9](#phase-9--rollout) |
+
+The gate in Phase 3 is the one to keep if any of this is ever rewritten: it found
+a rebuild that would have shipped payloads every receiver drops as forgeries,
+silently, one kind at a time.
+
 ## The constraint
 
 Two independent facts, and both have to be understood before the design makes
@@ -572,51 +590,94 @@ a member the first time they open a room they were added to late.
 
 **A day and a half, and do not skip it.**
 
-The unit tests are named in their phases. Three that only exist between devices,
+**They ran in the phases where the code they test first existed**, the way the
-all extending
+batch-signing note's did, so this section is the index rather than the work.
-[SignedGroupKeyStateTest](../composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/SignedGroupKeyStateTest.kt)'s
+Every claim below is asserted somewhere; what is here is which claim and where.
-harness -- two databases and the wire held by hand, with a third database added
-for the joiner:
 
-**The whole thing, end to end.** A and B run a real signing session over a
+**The whole thing, end to end** --
-chapter and its chunks. C, a database with no share, no `DkgSession`, no
+[ArchiveApplyJvmTest](../composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/ArchiveApplyJvmTest.kt),
-`FrostSigningSession` and no `GroupKeyState`, receives the archive and ends
+over two real databases with the pages carried by hand. The sender's room is
-holding rows identical to A's for every archived kind -- compared field by field,
+seeded through `ChatMessage.applyInnerEvent` itself, so what is archived is what
-signature included, not merely counted.
+a member's device really holds rather than rows built to suit the test. The
+receiver holds no share, no `DkgSession`, no `FrostSigningSession` and no
+`GroupKeyState`, and ends with the sender's rows.
 
-**The negative one that matters.** A hostile member sends C an archive of
+Compared as `(id, author, signature)` per row rather than by count, and then
-hand-built events: one with C's room id as `pubKey` and a random signature, one
+asserted that every archived row is authored by the room and carries a signature.
-validly signed by *another* room's key, one edited after signing, and one honest.
+Counting is not the claim: two databases can hold the same number of artifacts
-C ends with exactly one row. This is the test the feature's security is, and the
+and disagree about all of them, and a rebuild that lost the group's signature --
-only way to be a dishonest member in this harness is to build the inner event by
+or re-authored a row as whoever sent it -- would pass a count and fail the only
-hand rather than let a device queue it.
+thing this is for. The artifact version is the one exception and has to be:
+nobody signs it, it is derived from the signed artifact on arrival, which is why
+it is not archived and why a chapter's foreign key survives anyway.
 
-**The replay that must not work.** An archive carrying a genuine, still-valid
+**The negative one that matters** -- four ways to be a dishonest member in one
-`GroupKeyStateEvent` from an earlier epoch changes nothing about C's key state.
+page beside one honest dialect: the room's id as author with a made-up signature,
-It passes every signature check there is; the allowlist is the only thing that
+a real quorum of another group, an event edited after signing, and a member's own
-stops it, which is why it needs a test of its own rather than a line in the one
+rumor, which is what everything on the wire looks like today. The receiver ends
-above.
+with exactly the honest one. The only way to be dishonest in this harness is to
+build the inner event by hand rather than let a device queue it, which is what
+this does.
 
-And in `commonTest`, with no database: the `toXEvent` round-trip per archived
+**The replay that must not work** -- a genuine, still-verifying
-kind, asserting the reassembled event's id and that its stored signature still
+`GroupKeyStateEvent` in a hand-rolled page. It passes every signature check there
-verifies against it. That is the assumption Phase 3 rests on.
+is; the allowlist is the only thing that stops it, and the page has to be
+hand-rolled because `ArchiveEvent.build` refuses the kind, which is the outbound
+half of the same rule.
+
+**The `toXEvent` round trip per archived kind** --
+[ArchiveRoundTripTest](../composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveRoundTripTest.kt),
+against real FROST with no database. This is the one that earned its place: it
+found that `MantraArtifact.toArtifactEvent` had the alt tag in the wrong position
+*and* omitted the version metadata entirely, either of which produces payloads
+every receiver silently drops as forgeries. It also holds the negative -- a
+rebuild with the wrong version label fails as a forgery rather than as a mistake
+-- and a guard that the case list and `ARCHIVABLE_KINDS` move together.
+
+**And three that were not in the first draft**, each written because a test
+passed for the wrong reason or a bound could not fire:
+
+- *A page at exactly `MAX_PAGE_EVENTS` still decodes.* Without it the
+  page-over-the-cap test passes while the count cap is unreachable behind the
+  byte cap, which is how it was first written.
+- *Pages delivered backwards really did fail first.* Otherwise "out of order
+  converges" would pass on an archive that was never out of order, and the sweep
+  -- the only reason it converges -- would be untested.
+- *The page that completes an archive leaves nothing behind.* This is what caught
+  the sweep returning the sum of its passes rather than the settled one, which
+  made `failed > 0` stop meaning "still missing".
 
 ---
 
 ## Phase 9 -- rollout
 
-**No code.**
+**No code, and one constraint that is sharper than the first draft said.**
 
-Both kinds are new, so an old build receives an unknown inner event kind and
+The receiving half is safe to ship on its own, and phases 1-4 are exactly that:
-files it as unsupported, exactly as it does today for anything it does not know.
+nothing sends an archive until Phase 5 asks for one. That is the half to have in
-Nothing breaks in a mixed group; a joiner on an old build simply gets no archive,
+the field first.
-and one on a new build in a group of old builds gets no answer to its request.
-Neither is worse than today, which is no archive for anybody.
 
-The order is deliberate: Phases 1-4 are shippable together and do nothing on
+**Sending into a group with an old build is not free.** The first draft said an
-their own, because nothing sends an archive until Phase 5 asks for one. That
+old build "files it as unsupported, exactly as it does today for anything it does
-makes the first release a pure receiving capability, which is the safe half to
+not know", which is true and reads better than it lives. The unsupported row's
-have in the field first.
+content is `event.toJson()`, and it renders as an ordinary chat bubble -- so
+every member on an old build sees each archive page as a raw-JSON bubble of up to
+`MAX_PAGE_BYTES`, in a transcript, once per page.
+
+Nothing breaks and nothing is lost. But a group mid-upgrade gets a genuinely
+unpleasant transcript, and that is worth knowing before the first archive goes
+out rather than after. The rule:
+
+> Confirm every member is on a build that understands kind 30327 before any
+> member starts sending. There is no negotiation for this and adding one is not
+> worth it -- the cost of getting it wrong is ugly rather than dangerous, and it
+> stops as soon as they upgrade.
+
+The mitigation, if that ever proves unacceptable, is the one the appendix rejects
+for other reasons: carrying pages as Marmot direct messages, where an old build
+sees a gift wrap it cannot open and renders *"sent a private message"* with no
+content. It buys graceful degradation and costs everything listed under
+[Carrying the archive as a Marmot direct message](#appendix--what-was-considered-and-rejected).
 
 ---