feat: archive the translated text too, now that the group signs it

The merge brought in two commits that close the gap this feature was written
around, so the allowlist grows from six kinds to eight.

`feat: sign an artifact's first version with it, not derive it after` makes the
version the second item of the artifact's own signing batch. `feat: ask the group
to sign a chunk's translation, not just save it` puts a quorum behind the prose.
Both were done for their own reasons and neither was about the archive, but they
are exactly what the archive was missing: an archive can only carry what its
recipient can check, so a derived version and a member-authored translation could
not travel. A new member got the whole structure and none of the words.

**30301 and 30309 do not go on the end of the list.** The order is the foreign
keys: a version sits between its artifact and the chapters hanging off it, and a
translated chunk hangs off both a source chunk and a translation chapter, so that
one really is last.

**`toArtifactVersionEvent` had the bug this predicted it would.** It emitted
[artifactId, alt] where `build` emits [alt, artifactId], so the id did not
round-trip -- the same fault fixed on `MantraArtifact.toArtifactEvent` in Phase 3,
in the second of the three unused rebuilds, and for the same reason: nothing had
ever called it, so the "tag order matches build" claim in its comment was never
checked. `toTranslationChunkEvent` was already correct. Both now have a
round-trip case, which is what makes the difference between a rebuild that is
right and one that has not been contradicted yet.

**`signedEventsOf` walks two steps further**, emitting each version and the
translation chunks under each translation chapter. A retranslated passage
archives once: the arm that applies a translation chunk drops the one it
supersedes -- newest by the timestamp the group signed at, id breaking a tie --
so what a sender holds, and therefore what travels, is the group's current answer
to each passage rather than its drafts.

**The seeds had to change with it.** Both database tests derived the artifact's
first version by applying the artifact, which is exactly what stopped happening;
they now sign it through `ArtifactVersionEvent.initialVersionOf`, the way the
batch does. That also removes the one exception in the end-to-end assertion:
every archived row is now authored by the room and carries a signature, where the
artifact version used to have to be excused for having neither.

480 tests pass. The plan's Phase 3 table, its built-vs-plan table and its "what
this does not do" section are updated -- what an archive cannot do is down from
two things to one, and the remaining one is that it still cannot make its
recipient able to sign.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kgothatso NgakoandClaude Opus 5 committed 2026-09-06 15:13:43 +02:00
1 parent 54091099a9
commit 47aa79ebc7
8 files changed
+253 -130

No files matched your search

@@ -68,12 +68,16 @@ data class MantraArtifactVersion(
id = id, id = id,
pubKey = publicKey, pubKey = publicKey,
createdAt = createdAt.epochSeconds, createdAt = createdAt.epochSeconds,
// Tag order matches ArtifactVersionEvent.initialVersionOf, which puts
// the alt first because `build` does, or the event id does not
// round-trip. It was the other way round here for as long as nothing
// called this -- see ArchiveRoundTripTest, which is what calls it now.
tags = TagArrayBuilder<ArtifactVersionEvent>() tags = TagArrayBuilder<ArtifactVersionEvent>()
.addUnique( .addUnique(
ArtifactIdTag.assemble(artifactId) AltTag.assemble(ArtifactVersionEvent.ALT_DESCRIPTION)
) )
.addUnique( .addUnique(
AltTag.assemble(ArtifactVersionEvent.ALT_DESCRIPTION) ArtifactIdTag.assemble(artifactId)
) )
.build(), .build(),
content = versionLabel, content = versionLabel,
@@ -577,10 +577,12 @@ object ArchiveManager {
.getArtifactVersionsByArtifactId(artifact.id) .getArtifactVersionsByArtifactId(artifact.id)
// The version an artifact starts life with carries the label the // The version an artifact starts life with carries the label the
// artifact was signed with, and `initialVersionOf` gives it the // artifact was signed with, and `ArtifactVersionEvent.initialVersionOf`
// artifact's own timestamp. A row with none of those is an artifact // gives it the artifact's own timestamp. The label is still not on the
// this device cannot rebuild, which is a gap in the archive rather // artifact row -- `fromArtifactEvent` drops it -- so this is where it
// than a reason to abandon it. // comes back from. An artifact with no such version is one this device
// cannot rebuild, which is a gap in the archive rather than a reason to
// abandon it.
val versionLabel = versions val versionLabel = versions
.firstOrNull { it.createdAt == artifact.createdAt } .firstOrNull { it.createdAt == artifact.createdAt }
?.versionLabel ?.versionLabel
@@ -592,6 +594,8 @@ object ArchiveManager {
} }
versions.forEach { version -> versions.forEach { version ->
add(version.toArtifactVersionEvent())
database.mantraChapterDao() database.mantraChapterDao()
.getChaptersByArtifactVersionId(version.id) .getChaptersByArtifactVersionId(version.id)
.forEach { chapter -> .forEach { chapter ->
@@ -609,7 +613,20 @@ object ArchiveManager {
database.mantraTranslationChapterDao() database.mantraTranslationChapterDao()
.getTranslationChaptersByTranslationArtifactVersionId(translationVersion.id) .getTranslationChaptersByTranslationArtifactVersionId(translationVersion.id)
.forEach { add(it.toTranslationChapterEvent()) } .forEach { translationChapter ->
add(translationChapter.toTranslationChapterEvent())
// Only the translation that survived supersession
// is here to be found: retranslating a passage
// changes the text and so the event id, and the
// arm that applies one drops what it replaces. So
// an archive carries a group's current answer to
// each passage rather than its drafts, which is
// the same thing every other member holds.
database.mantraTranslationChunkDao()
.getTranslationChunksByTranslationChapterId(translationChapter.id)
.forEach { add(it.toTranslationChunkEvent()) }
}
} }
} }
} }
@@ -15,11 +15,13 @@ import press.mantra.compose.nostr.archive.tags.ArchiveIdTag
import press.mantra.compose.nostr.archive.tags.ArchivePageTag import press.mantra.compose.nostr.archive.tags.ArchivePageTag
import press.mantra.compose.nostr.archive.tags.ArchiveRecipientTag import press.mantra.compose.nostr.archive.tags.ArchiveRecipientTag
import press.mantra.compose.nostr.nip30303.ArtifactEvent import press.mantra.compose.nostr.nip30303.ArtifactEvent
import press.mantra.compose.nostr.nip30303.ArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.ChapterEvent import press.mantra.compose.nostr.nip30303.ChapterEvent
import press.mantra.compose.nostr.nip30303.ChunkEvent import press.mantra.compose.nostr.nip30303.ChunkEvent
import press.mantra.compose.nostr.nip30303.DialectEvent import press.mantra.compose.nostr.nip30303.DialectEvent
import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.TranslationChapterEvent import press.mantra.compose.nostr.nip30303.TranslationChapterEvent
import press.mantra.compose.nostr.nip30303.TranslationChunkEvent
/** /**
* 30327 * 30327
@@ -170,35 +172,32 @@ class ArchiveEvent(
* rumors -- empty signature, member author, vouched for by the MLS frame * rumors -- empty signature, member author, vouched for by the MLS frame
* they arrived in and by nothing that survives leaving it. * they arrived in and by nothing that survives leaving it.
* *
* Left out for that reason, and this is a real limit rather than an * Left out for that reason:
* oversight:
* *
* - `ArtifactVersionEvent` (30301). Never signed. The version an
* artifact starts life with is *derived* from the signed artifact by
* `MantraArtifactVersion.initialVersionOf`, which the artifact's own
* arm in `applyInnerEvent` runs -- so archiving the artifact brings its
* version along and the chapters hanging off it keep their foreign
* key. Later versions go through `MantraDao.addArtifactVersion` as
* rumors, and no screen calls it today.
* - `TranslationChunkEvent` (30309) -- **the translated text itself**.
* `MantraDao.saveTranslation` submits it as a member's rumor, because
* a translation is one member's work rather than a group decision.
* So an archive restores everything a translation hangs on and not the
* translation. See docs/member-archive.md for what closing that would
* take; it is not a line in this list.
* - `TranslationEvent` (30311). Nothing builds one; the inbound arm * - `TranslationEvent` (30311). Nothing builds one; the inbound arm
* exists and no producer does. * exists and no producer does.
* - The contributor lists (30305, 30307, 30310). `applyInnerEvent` has * - The contributor lists (30305, 30307, 30310). `applyInnerEvent` has
* no arm that writes a row for any of them, so archiving them would * no arm that writes a row for any of them, so archiving them would
* cost bytes and restore nothing. * cost bytes and restore nothing.
*
* Two kinds were on that list and are not any more, because the app
* stopped leaving them unsigned. `ArtifactVersionEvent` (30301) used to be
* derived from the signed artifact on arrival, and is now the second item
* of the batch that carries it. `TranslationChunkEvent` (30309) -- the
* translated text itself -- used to be submitted as its author's rumor,
* and is now proposed to the group like everything else. Both are
* therefore checkable, and an archive that left them out would hand a new
* member the whole structure and none of the prose.
*/ */
private val APPLY_ORDER: List<Kind> = listOf( private val APPLY_ORDER: List<Kind> = listOf(
DialectEvent.KIND, DialectEvent.KIND,
ArtifactEvent.KIND, ArtifactEvent.KIND,
ArtifactVersionEvent.KIND,
ChapterEvent.KIND, ChapterEvent.KIND,
ChunkEvent.KIND, ChunkEvent.KIND,
TranslationArtifactVersionEvent.KIND, TranslationArtifactVersionEvent.KIND,
TranslationChapterEvent.KIND, TranslationChapterEvent.KIND,
TranslationChunkEvent.KIND,
) )
/** Every kind an archive may carry. */ /** Every kind an archive may carry. */
@@ -183,33 +183,37 @@ class ArchiveEventTest {
listOf( listOf(
DialectEvent.KIND, DialectEvent.KIND,
ArtifactEvent.KIND, ArtifactEvent.KIND,
ArtifactVersionEvent.KIND,
ChapterEvent.KIND, ChapterEvent.KIND,
ChunkEvent.KIND, ChunkEvent.KIND,
TranslationArtifactVersionEvent.KIND, TranslationArtifactVersionEvent.KIND,
TranslationChapterEvent.KIND, TranslationChapterEvent.KIND,
TranslationChunkEvent.KIND,
).forEach { assertTrue(ArchiveEvent.isArchivable(it), "kind $it should be archivable") } ).forEach { assertTrue(ArchiveEvent.isArchivable(it), "kind $it should be archivable") }
assertEquals(6, ArchiveEvent.ARCHIVABLE_KINDS.size) assertEquals(8, ArchiveEvent.ARCHIVABLE_KINDS.size)
} }
@Test @Test
fun `the kinds nobody signs are left out, translations included`() { fun `the translated text is archivable, now that the group signs it`() {
// The limit worth knowing about before reading anything else here. An // Worth its own case because it was the feature's headline limitation for
// archive can only carry what the receiver can check, and six of the // as long as `saveTranslation` wrote the row and queued the member's own
// thirteen nip30303 kinds reach a signing session. The rest travel as // rumor. An archive can only carry what the receiver can check, so a
// rumors -- empty signature, member author -- vouched for by the MLS // translation nobody signed could not travel, and a new member got the
// frame they arrived in and by nothing that survives leaving it. // whole structure and none of the prose.
assertTrue(ArchiveEvent.isArchivable(TranslationChunkEvent.KIND))
// The translated text. MantraDao.saveTranslation submits it as the // Same shape, same fix: the version an artifact starts life with used to
// member's own rumor, so an archive restores everything a translation // be derived on arrival, so it was authored by the group with no
// hangs on and not the translation. // signature to show for it. It is the second item of the artifact's batch
assertFalse(ArchiveEvent.isArchivable(TranslationChunkEvent.KIND)) // now.
assertTrue(ArchiveEvent.isArchivable(ArtifactVersionEvent.KIND))
}
// Never signed either: derived from the signed artifact on arrival, which @Test
// is what keeps a chapter's foreign key satisfied without archiving it. fun `the kinds nobody signs are still left out`() {
assertFalse(ArchiveEvent.isArchivable(ArtifactVersionEvent.KIND)) // Nothing builds one of these at all; the inbound arm exists and no
// producer does.
// Nothing builds one of these at all.
assertFalse(ArchiveEvent.isArchivable(TranslationEvent.KIND)) assertFalse(ArchiveEvent.isArchivable(TranslationEvent.KIND))
} }
@@ -253,18 +257,18 @@ class ArchiveEventTest {
// Every one of these is a foreign key in Room, so an archive applied the // Every one of these is a foreign key in Room, so an archive applied the
// other way round is a constraint violation rather than a wrong answer. // other way round is a constraint violation rather than a wrong answer.
assertTrue(rank(DialectEvent.KIND) < rank(ArtifactEvent.KIND)) assertTrue(rank(DialectEvent.KIND) < rank(ArtifactEvent.KIND))
assertTrue(rank(ArtifactEvent.KIND) < rank(ArtifactVersionEvent.KIND))
assertTrue(rank(ArtifactVersionEvent.KIND) < rank(ChapterEvent.KIND))
assertTrue(rank(ChapterEvent.KIND) < rank(ChunkEvent.KIND)) assertTrue(rank(ChapterEvent.KIND) < rank(ChunkEvent.KIND))
assertTrue(rank(ArtifactVersionEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND))
assertTrue(rank(DialectEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND))
assertTrue(rank(TranslationArtifactVersionEvent.KIND) < rank(TranslationChapterEvent.KIND)) assertTrue(rank(TranslationArtifactVersionEvent.KIND) < rank(TranslationChapterEvent.KIND))
assertTrue(rank(ChapterEvent.KIND) < rank(TranslationChapterEvent.KIND)) assertTrue(rank(ChapterEvent.KIND) < rank(TranslationChapterEvent.KIND))
assertTrue(rank(DialectEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND))
// A chapter and a translation artifact version both hang off an artifact // A translated chunk hangs off both sides at once -- the source chunk it
// *version*, which is not archived: it is derived from the signed // translates and the translation chapter it sits in -- so it is last.
// artifact by the artifact's own arm in applyInnerEvent. So the artifact assertTrue(rank(ChunkEvent.KIND) < rank(TranslationChunkEvent.KIND))
// has to land before either of them, and the dependency runs through a assertTrue(rank(TranslationChapterEvent.KIND) < rank(TranslationChunkEvent.KIND))
// kind that is not in this list at all.
assertTrue(rank(ArtifactEvent.KIND) < rank(ChapterEvent.KIND))
assertTrue(rank(ArtifactEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND))
// And the numbers really do disagree with the order, which is why this is // And the numbers really do disagree with the order, which is why this is
// a list rather than a sortedBy { kind }. // a list rather than a sortedBy { kind }.
@@ -275,11 +279,13 @@ class ArchiveEventTest {
@Test @Test
fun `sorting a page puts what is referenced before what refers to it`() { fun `sorting a page puts what is referenced before what refers to it`() {
val jumbled = listOf( val jumbled = listOf(
payload(TranslationChunkEvent.KIND),
payload(TranslationChapterEvent.KIND), payload(TranslationChapterEvent.KIND),
payload(ChunkEvent.KIND), payload(ChunkEvent.KIND),
payload(DialectEvent.KIND), payload(DialectEvent.KIND),
payload(TranslationArtifactVersionEvent.KIND), payload(TranslationArtifactVersionEvent.KIND),
payload(ChapterEvent.KIND), payload(ChapterEvent.KIND),
payload(ArtifactVersionEvent.KIND),
payload(ArtifactEvent.KIND), payload(ArtifactEvent.KIND),
) )
@@ -287,10 +293,12 @@ class ArchiveEventTest {
listOf( listOf(
DialectEvent.KIND, DialectEvent.KIND,
ArtifactEvent.KIND, ArtifactEvent.KIND,
ArtifactVersionEvent.KIND,
ChapterEvent.KIND, ChapterEvent.KIND,
ChunkEvent.KIND, ChunkEvent.KIND,
TranslationArtifactVersionEvent.KIND, TranslationArtifactVersionEvent.KIND,
TranslationChapterEvent.KIND, TranslationChapterEvent.KIND,
TranslationChunkEvent.KIND,
), ),
ArchiveEvent.inApplyOrder(jumbled).map { it.kind } ArchiveEvent.inApplyOrder(jumbled).map { it.kind }
) )
@@ -19,20 +19,24 @@ import kotlin.test.assertFalse
import kotlin.test.assertNotNull import kotlin.test.assertNotNull
import kotlin.test.assertTrue import kotlin.test.assertTrue
import press.mantra.compose.database.model.MantraArtifact import press.mantra.compose.database.model.MantraArtifact
import press.mantra.compose.database.model.MantraArtifactVersion
import press.mantra.compose.database.model.MantraChapter import press.mantra.compose.database.model.MantraChapter
import press.mantra.compose.database.model.MantraChunk import press.mantra.compose.database.model.MantraChunk
import press.mantra.compose.database.model.MantraDialect import press.mantra.compose.database.model.MantraDialect
import press.mantra.compose.database.model.MantraTranslationArtifactVersion import press.mantra.compose.database.model.MantraTranslationArtifactVersion
import press.mantra.compose.database.model.MantraTranslationChapter import press.mantra.compose.database.model.MantraTranslationChapter
import press.mantra.compose.database.model.MantraTranslationChunk
import press.mantra.compose.extensions.toHex import press.mantra.compose.extensions.toHex
import press.mantra.compose.managers.SharedKeyDerivation import press.mantra.compose.managers.SharedKeyDerivation
import press.mantra.compose.nostr.frost.GroupKeyStateEvent import press.mantra.compose.nostr.frost.GroupKeyStateEvent
import press.mantra.compose.nostr.nip30303.ArtifactEvent import press.mantra.compose.nostr.nip30303.ArtifactEvent
import press.mantra.compose.nostr.nip30303.ArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.ChapterEvent import press.mantra.compose.nostr.nip30303.ChapterEvent
import press.mantra.compose.nostr.nip30303.ChunkEvent import press.mantra.compose.nostr.nip30303.ChunkEvent
import press.mantra.compose.nostr.nip30303.DialectEvent import press.mantra.compose.nostr.nip30303.DialectEvent
import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.TranslationChapterEvent import press.mantra.compose.nostr.nip30303.TranslationChapterEvent
import press.mantra.compose.nostr.nip30303.TranslationChunkEvent
/** /**
* The assumption the whole archive rests on: a row can be turned back into the * The assumption the whole archive rests on: a row can be turned back into the
@@ -79,6 +83,8 @@ class ArchiveRoundTripTest {
private val artifactVersionId = "c".repeat(64) private val artifactVersionId = "c".repeat(64)
private val chapterId = "d".repeat(64) private val chapterId = "d".repeat(64)
private val translationArtifactVersionId = "e".repeat(64) private val translationArtifactVersionId = "e".repeat(64)
private val translationChapterId = "f".repeat(64)
private val chunkId = "a".repeat(64)
/** Exactly what `FrostSigningManager.unsignedEventOf` does. */ /** Exactly what `FrostSigningManager.unsignedEventOf` does. */
private fun unsignedEventOf(template: EventTemplate<*>) = Event( private fun unsignedEventOf(template: EventTemplate<*>) = Event(
@@ -258,6 +264,63 @@ class ArchiveRoundTripTest {
assertFalse(GroupKeyStateEvent.isSignedByRoom(rebuilt, chatRoomId)) assertFalse(GroupKeyStateEvent.isSignedByRoom(rebuilt, chatRoomId))
} }
@Test
fun `an artifact version survives the trip through a row`() {
// The second fault of the same shape, found the moment this kind became
// archivable: `toArtifactVersionEvent` emitted [artifactId, alt] where
// `build` emits [alt, artifactId]. It had been that way for as long as
// nothing called it, which is what makes an unused rebuild dangerous
// rather than merely dead -- the claim in its comment was never checked.
//
// Built through `initialVersionOf` rather than by hand, because that is
// what the signing batch actually proposes.
val artifact = signed(
ArtifactEvent.build(
name = "In Detention",
url = "https://example.com/in-detention",
visibility = "private",
license = "cc",
dialectId = dialectId,
versionLabel = "1.0",
createdAt = 1_700_000_000L
)
)
assertRoundTrips(
ArtifactVersionEvent.initialVersionOf(artifact).single()
) { event ->
MantraArtifactVersion.fromArtifactVersionEvent(
ArtifactVersionEvent(
event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig
),
chatRoomId
)?.toArtifactVersionEvent()
}
}
@Test
fun `a translated chunk survives the trip through a row`() {
// The translated text. Archivable only because the app stopped saving it
// as its author's rumor and started asking the group to sign it -- until
// then a new member got the whole structure and none of the prose.
assertRoundTrips(
TranslationChunkEvent.build(
translationChapterId = translationChapterId,
chunkId = chunkId,
index = 0,
text = "Wayeyindoda enezingxenye.",
createdAt = 1_700_000_000L
)
) { event ->
MantraTranslationChunk.fromTranslationChunkEvent(
TranslationChunkEvent(
event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig
),
chatRoomId
)?.toTranslationChunkEvent()
}
}
@Test @Test
fun `a chapter survives the trip through a row`() { fun `a chapter survives the trip through a row`() {
assertRoundTrips( assertRoundTrips(
@@ -343,7 +406,7 @@ class ArchiveRoundTripTest {
// allowlist without a case here is one nobody has checked can be rebuilt, // allowlist without a case here is one nobody has checked can be rebuilt,
// and the failure is silent: the receiver drops it as a forgery. // and the failure is silent: the receiver drops it as a forgery.
assertEquals( assertEquals(
6, 8,
ArchiveEvent.ARCHIVABLE_KINDS.size, ArchiveEvent.ARCHIVABLE_KINDS.size,
"an archivable kind was added or removed; add or remove its round-trip case" "an archivable kind was added or removed; add or remove its round-trip case"
) )
@@ -40,11 +40,13 @@ import press.mantra.compose.nostr.archive.tags.ArchiveRecipientTag
import com.vitorpamplona.quartz.marmot.mip02Welcome.WelcomeEvent import com.vitorpamplona.quartz.marmot.mip02Welcome.WelcomeEvent
import press.mantra.compose.nostr.frost.GroupKeyStateEvent import press.mantra.compose.nostr.frost.GroupKeyStateEvent
import press.mantra.compose.nostr.nip30303.ArtifactEvent import press.mantra.compose.nostr.nip30303.ArtifactEvent
import press.mantra.compose.nostr.nip30303.ArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.ChapterEvent import press.mantra.compose.nostr.nip30303.ChapterEvent
import press.mantra.compose.nostr.nip30303.ChunkEvent import press.mantra.compose.nostr.nip30303.ChunkEvent
import press.mantra.compose.nostr.nip30303.DialectEvent import press.mantra.compose.nostr.nip30303.DialectEvent
import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.TranslationChapterEvent import press.mantra.compose.nostr.nip30303.TranslationChapterEvent
import press.mantra.compose.nostr.nip30303.TranslationChunkEvent
/** /**
* Two devices, two databases: one that did the work and one that arrived after * Two devices, two databases: one that did the work and one that arrived after
@@ -229,9 +231,10 @@ class ArchiveApplyJvmTest {
) )
).also { apply(sender, it) } ).also { apply(sender, it) }
val version = assertNotNull( // The second item of the artifact's own signing batch, not a row derived
sender.mantraArtifactVersionDao().getArtifactVersionsByArtifactId(artifact.id).firstOrNull() // on arrival -- which is what makes it archivable at all.
) val version = signed(ArtifactVersionEvent.initialVersionOf(artifact).single())
.also { apply(sender, it) }
val chapter = signed( val chapter = signed(
ChapterEvent.build( ChapterEvent.build(
@@ -245,19 +248,16 @@ class ArchiveApplyJvmTest {
) )
).also { apply(sender, it) } ).also { apply(sender, it) }
apply( val chunk = signed(
sender, ChunkEvent.build(
signed( chapterId = chapter.id,
ChunkEvent.build( text = "He was a man of parts.",
chapterId = chapter.id, index = 0,
text = "He was a man of parts.", wordCount = 6,
index = 0, characterCount = 22,
wordCount = 6, createdAt = 1_700_000_030L
characterCount = 22,
createdAt = 1_700_000_030L
)
) )
) ).also { apply(sender, it) }
apply( apply(
sender, sender,
signed( signed(
@@ -283,14 +283,27 @@ class ArchiveApplyJvmTest {
) )
).also { apply(sender, it) } ).also { apply(sender, it) }
val translationChapter = signed(
TranslationChapterEvent.build(
translationArtifactVersionId = translationVersion.id,
chapterId = chapter.id,
index = 0,
createdAt = 1_700_000_050L
)
).also { apply(sender, it) }
// The translated text. Group-signed now, so it can travel and be checked
// -- until it was, an archive handed a new member the whole structure and
// none of the prose.
apply( apply(
sender, sender,
signed( signed(
TranslationChapterEvent.build( TranslationChunkEvent.build(
translationArtifactVersionId = translationVersion.id, translationChapterId = translationChapter.id,
chapterId = chapter.id, chunkId = chunk.id,
index = 0, index = 0,
createdAt = 1_700_000_050L text = "Wayeyindoda enezingxenye.",
createdAt = 1_700_000_060L
) )
) )
) )
@@ -356,6 +369,9 @@ class ArchiveApplyJvmTest {
db.mantraTranslationChapterDao() db.mantraTranslationChapterDao()
.getTranslationChaptersByTranslationArtifactVersionId(it.id) .getTranslationChaptersByTranslationArtifactVersionId(it.id)
} }
val translationChunks = translationChapters.flatMap {
db.mantraTranslationChunkDao().getTranslationChunksByTranslationChapterId(it.id)
}
return mapOf( return mapOf(
"dialects" to dialects.size, "dialects" to dialects.size,
@@ -365,6 +381,7 @@ class ArchiveApplyJvmTest {
"chunks" to chunks.size, "chunks" to chunks.size,
"translationVersions" to translationVersions.size, "translationVersions" to translationVersions.size,
"translationChapters" to translationChapters.size, "translationChapters" to translationChapters.size,
"translationChunks" to translationChunks.size,
) )
} }
@@ -395,11 +412,13 @@ class ArchiveApplyJvmTest {
chapters.flatMap { db.mantraChunkDao().getChunksByChapterId(it.id) } chapters.flatMap { db.mantraChunkDao().getChunksByChapterId(it.id) }
.forEach { add(Triple(it.id, it.publicKey, it.signature)) } .forEach { add(Triple(it.id, it.publicKey, it.signature)) }
translationVersions.forEach { add(Triple(it.id, it.publicKey, it.signature)) } translationVersions.forEach { add(Triple(it.id, it.publicKey, it.signature)) }
translationVersions val translationChapters = translationVersions.flatMap {
.flatMap { db.mantraTranslationChapterDao()
db.mantraTranslationChapterDao() .getTranslationChaptersByTranslationArtifactVersionId(it.id)
.getTranslationChaptersByTranslationArtifactVersionId(it.id) }
} translationChapters.forEach { add(Triple(it.id, it.publicKey, it.signature)) }
translationChapters
.flatMap { db.mantraTranslationChunkDao().getTranslationChunksByTranslationChapterId(it.id) }
.forEach { add(Triple(it.id, it.publicKey, it.signature)) } .forEach { add(Triple(it.id, it.publicKey, it.signature)) }
}.sortedBy { it.first } }.sortedBy { it.first }
} }
@@ -413,6 +432,7 @@ class ArchiveApplyJvmTest {
mapOf( mapOf(
"dialects" to 0, "artifacts" to 0, "versions" to 0, "chapters" to 0, "dialects" to 0, "artifacts" to 0, "versions" to 0, "chapters" to 0,
"chunks" to 0, "translationVersions" to 0, "translationChapters" to 0, "chunks" to 0, "translationVersions" to 0, "translationChapters" to 0,
"translationChunks" to 0,
), ),
rowCounts(receiver), rowCounts(receiver),
"the new member starts with nothing, which is the whole problem" "the new member starts with nothing, which is the whole problem"
@@ -429,16 +449,11 @@ class ArchiveApplyJvmTest {
val fingerprints = rowFingerprints(receiver) val fingerprints = rowFingerprints(receiver)
assertEquals(rowFingerprints(sender), fingerprints) assertEquals(rowFingerprints(sender), fingerprints)
// And every one of them is the room's own work rather than anybody's. // And every one of them is the room's own work, with no exception left.
// The artifact version is the exception and has to be: nobody signs it, // The artifact version used to be one -- derived on arrival, so authored
// it is derived from the signed artifact on arrival, which is exactly why // by the group and carrying no signature to show for it -- and is signed
// it is not archived and why a chapter's foreign key survives anyway. // in the artifact's own batch now, which is what let it into the archive.
val versionIds = receiver.mantraArtifactDao().getArtifactsByChatRoomId(chatRoomId) fingerprints.forEach { (id, author, signature) ->
.flatMap { receiver.mantraArtifactVersionDao().getArtifactVersionsByArtifactId(it.id) }
.map { it.id }
.toSet()
fingerprints.filterNot { it.first in versionIds }.forEach { (id, author, signature) ->
assertEquals(chatRoomId, author, "row $id is not authored by the room") assertEquals(chatRoomId, author, "row $id is not authored by the room")
assertTrue(signature.isNotBlank(), "row $id came across without a signature") assertTrue(signature.isNotBlank(), "row $id came across without a signature")
} }
@@ -625,6 +640,7 @@ class ArchiveApplyJvmTest {
mapOf( mapOf(
"dialects" to 0, "artifacts" to 0, "versions" to 0, "chapters" to 0, "dialects" to 0, "artifacts" to 0, "versions" to 0, "chapters" to 0,
"chunks" to 0, "translationVersions" to 0, "translationChapters" to 0, "chunks" to 0, "translationVersions" to 0, "translationChapters" to 0,
"translationChunks" to 0,
), ),
rowCounts(receiver), rowCounts(receiver),
"a bystander already holds the work; re-applying would only rewrite its provenance" "a bystander already holds the work; re-applying would only rewrite its provenance"
@@ -31,11 +31,13 @@ import press.mantra.compose.extensions.toHex
import press.mantra.compose.nostr.archive.ArchiveEvent import press.mantra.compose.nostr.archive.ArchiveEvent
import press.mantra.compose.nostr.frost.GroupKeyStateEvent import press.mantra.compose.nostr.frost.GroupKeyStateEvent
import press.mantra.compose.nostr.nip30303.ArtifactEvent import press.mantra.compose.nostr.nip30303.ArtifactEvent
import press.mantra.compose.nostr.nip30303.ArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.ChapterEvent import press.mantra.compose.nostr.nip30303.ChapterEvent
import press.mantra.compose.nostr.nip30303.ChunkEvent import press.mantra.compose.nostr.nip30303.ChunkEvent
import press.mantra.compose.nostr.nip30303.DialectEvent import press.mantra.compose.nostr.nip30303.DialectEvent
import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.TranslationChapterEvent import press.mantra.compose.nostr.nip30303.TranslationChapterEvent
import press.mantra.compose.nostr.nip30303.TranslationChunkEvent
/** /**
* Assembling a room's archive out of the rows a device actually holds. * Assembling a room's archive out of the rows a device actually holds.
@@ -195,12 +197,10 @@ class ArchiveAssemblyJvmTest {
) )
) )
// Derived from the artifact rather than signed, which is exactly why it // The second item of the artifact's own batch, built the way the batch
// is not archived and why its label has to be read back off this row. // builds it. It used to be derived on arrival, which is why the archive
val version = assertNotNull( // once could not carry it: a derived row has no signature to check.
db.mantraArtifactVersionDao().getArtifactVersionsByArtifactId(artifact.id).firstOrNull(), val version = apply(ArtifactVersionEvent.initialVersionOf(artifact).single())
"applying the artifact should have derived its first version"
)
val chapter = apply( val chapter = apply(
ChapterEvent.build( ChapterEvent.build(
@@ -214,7 +214,7 @@ class ArchiveAssemblyJvmTest {
) )
) )
apply( val chunk = apply(
ChunkEvent.build( ChunkEvent.build(
chapterId = chapter.id, chapterId = chapter.id,
text = "He was a man of parts.", text = "He was a man of parts.",
@@ -246,7 +246,7 @@ class ArchiveAssemblyJvmTest {
) )
) )
apply( val translationChapter = apply(
TranslationChapterEvent.build( TranslationChapterEvent.build(
translationArtifactVersionId = translationVersion.id, translationArtifactVersionId = translationVersion.id,
chapterId = chapter.id, chapterId = chapter.id,
@@ -254,6 +254,18 @@ class ArchiveAssemblyJvmTest {
createdAt = 1_700_000_050L createdAt = 1_700_000_050L
) )
) )
// The translated text, which the group signs now rather than each member
// saving their own.
apply(
TranslationChunkEvent.build(
translationChapterId = translationChapter.id,
chunkId = chunk.id,
index = 0,
text = "Wayeyindoda enezingxenye.",
createdAt = 1_700_000_060L
)
)
} }
private suspend fun archivedPayloads(): List<Event> = private suspend fun archivedPayloads(): List<Event> =
@@ -283,10 +295,12 @@ class ArchiveAssemblyJvmTest {
assertEquals(1, byKind[DialectEvent.KIND]?.size) assertEquals(1, byKind[DialectEvent.KIND]?.size)
assertEquals(1, byKind[ArtifactEvent.KIND]?.size) assertEquals(1, byKind[ArtifactEvent.KIND]?.size)
assertEquals(1, byKind[ArtifactVersionEvent.KIND]?.size)
assertEquals(1, byKind[ChapterEvent.KIND]?.size) assertEquals(1, byKind[ChapterEvent.KIND]?.size)
assertEquals(2, byKind[ChunkEvent.KIND]?.size) assertEquals(2, byKind[ChunkEvent.KIND]?.size)
assertEquals(1, byKind[TranslationArtifactVersionEvent.KIND]?.size) assertEquals(1, byKind[TranslationArtifactVersionEvent.KIND]?.size)
assertEquals(1, byKind[TranslationChapterEvent.KIND]?.size) assertEquals(1, byKind[TranslationChapterEvent.KIND]?.size)
assertEquals(1, byKind[TranslationChunkEvent.KIND]?.size)
assertEquals(ArchiveEvent.ARCHIVABLE_KINDS, byKind.keys) assertEquals(ArchiveEvent.ARCHIVABLE_KINDS, byKind.keys)
} }
+43 -41
View File
@@ -17,7 +17,7 @@ times worth reading:
| what the plan said | what it turned out to be | | what the plan said | what it turned out to be |
|---|---| |---|---|
| nine archivable kinds | six. Only six ever reach a signing session, and one of the three that do not is the translated text -- see [Phase 3](#what-is-actually-archivable-which-is-less-than-it-looks) | | nine archivable kinds | six at first, eight now. The three that were unsigned were fixed in the app rather than worked around here -- see [Phase 3](#what-is-actually-archivable) |
| `MAX_PAGE_EVENTS = 256` | 128. At 256 the byte cap always binds first and the count cap can never fire | | `MAX_PAGE_EVENTS = 256` | 128. At 256 the byte cap always binds first and the count cap can never fire |
| "assemble, order, pack and queue" | assemble only; queueing moved to Phase 5, next to the thing that decides when | | "assemble, order, pack and queue" | assemble only; queueing moved to Phase 5, next to the thing that decides when |
| "re-read the room between the invite and the assembly" | unnecessary; that rule is about the MLS snapshot a commit is built on | | "re-read the room between the invite and the assembly" | unnecessary; that rule is about the MLS snapshot a commit is built on |
@@ -321,27 +321,41 @@ Neither fault would have shown up as an error. Both produce a well-formed
artifact whose id no longer matches its fields, which every receiver drops as a artifact whose id no longer matches its fields, which every receiver drops as a
forgery, silently, one kind at a time. forgery, silently, one kind at a time.
### What is actually archivable, which is less than it looks ### What is actually archivable
Only six of the thirteen nip30303 kinds ever reach a signing session. The rest An archive can only carry what its receiver can check, so the list is exactly the
travel as rumors -- empty signature, member author, vouched for by the MLS frame kinds a signing session produces. Eight of the thirteen nip30303 kinds do.
they arrived in and by nothing that survives leaving it -- so they cannot be put
in front of somebody who has no way to check them.
| kind | | why | | kind | | why |
|---|---|---| |---|---|---|
| 30304 Dialect, 30300 Artifact, 30302 Chapter, 30303 Chunk, 30306 TranslationArtifactVersion, 30308 TranslationChapter | archivable | proposed through `proposeSigning`/`proposeSigningBatch` | | 30304 Dialect, 30300 Artifact, 30301 ArtifactVersion, 30302 Chapter, 30303 Chunk, 30306 TranslationArtifactVersion, 30308 TranslationChapter, 30309 TranslationChunk | archivable | proposed through `proposeSigning` / `proposeSigningBatch` |
| 30301 ArtifactVersion | no | never signed; derived from the signed artifact on arrival, which is what keeps a chapter's foreign key satisfied without archiving it |
| 30309 TranslationChunk | no | **the translated text itself.** `MantraDao.saveTranslation` submits it as the member's own rumor |
| 30311 Translation | no | nothing builds one; the inbound arm exists and no producer does | | 30311 Translation | no | nothing builds one; the inbound arm exists and no producer does |
| 30305, 30307, 30310 contributor lists | no | `applyInnerEvent` has no arm that writes a row for any of them | | 30305, 30307, 30310 contributor lists | no | `applyInnerEvent` has no arm that writes a row for any of them |
The second row of "no" is the one that matters and it is not a detail: **an **It was six when this was written, and the two that were missing were the two
archive restores everything a translation hangs on and not the translation.** A that mattered.** An artifact version was derived from the signed artifact on
new member gets the dialects, the artifacts, the chapters, the source chunks, arrival -- a row naming the group as its author with no signature to show for it
which translations exist and their chapter scaffolding -- the whole structure, -- and a translated chunk was submitted as its author's rumor by
enough to start translating -- and none of the translated text. See `MantraDao.saveTranslation`. Neither could be put in front of somebody with no
[What this does not do](#what-this-does-not-do). way to check it, so an archive restored everything a translation hangs on and not
the translation.
Both were fixed in the app rather than worked around here, in parallel with this
work and for their own reasons: `feat: sign an artifact's first version with it,
not derive it after` makes the version the second item of the artifact's batch,
and `feat: ask the group to sign a chunk's translation, not just save it` puts a
quorum behind the prose. Once each of them carried a signature there was nothing
left to argue about -- the allowlist grew by two and the caveat went away.
The order is forced by the foreign keys, and 30301 and 30309 do not go on the
end: a version sits between its artifact and the chapters hanging off it, and a
translated chunk hangs off both a source chunk and a translation chapter, so it
really is last.
**A retranslated passage archives once.** The arm that applies a translation
chunk drops the one it supersedes -- newest by the timestamp the group signed at,
id breaking a tie -- so a sender holds a group's current answer to each passage
rather than its drafts, and that is what travels.
### Ordering ### Ordering
@@ -579,10 +593,11 @@ left out rather than written blind:
both real paths are automatic, so this is for the case the automation misses, both real paths are automatic, so this is for the case the automation misses,
and it wants a screen to live on. and it wants a screen to live on.
**Say what the new member cannot do.** Still unwritten, and still the part of **Say what the new member cannot do.** Still unwritten, and now down to one
this most likely to be reported as a bug: an archive does not make its recipient thing rather than two: an archive hands its recipient the group's whole signed
able to sign, and it does not carry the translated text. Both belong in front of record, prose included, and does not make them able to *sign* anything. That is
a member the first time they open a room they were added to late. the sentence a member wants the first time they open a room they were added to
late, and the first thing this will be reported as a bug for.
--- ---
@@ -715,28 +730,15 @@ general answer for the reason
[frost-batch-signing.md](./frost-batch-signing.md#appendix--what-was-considered-and-rejected) [frost-batch-signing.md](./frost-batch-signing.md#appendix--what-was-considered-and-rejected)
gives for manifests. Worth revisiting once anything depends on completeness. gives for manifests. Worth revisiting once anything depends on completeness.
**The translated text is not archived.** The largest gap, and it follows from the **Nothing unsigned is archived, and that is the whole list.** For a while it read
same rule everything else here follows from. A translation chunk is submitted by larger: the translated text was its author's rumor and an artifact's first
`MantraDao.saveTranslation` as the member's own rumor, because a translation is version was derived rather than signed, so neither could travel and a new member
one member's work rather than a group decision -- so it carries no signature, and got the structure and none of the prose. Both are signed now. What is left out is
an archive carrying it would be asking its recipient to believe whoever sent it. `TranslationEvent`, which nothing builds, and the contributor lists, which
A new member therefore receives the entire structure and none of the prose. nothing applies -- so the rule and the list have stopped diverging, and the thing
to watch is that they do not drift apart again. The guard is
Three ways out, in increasing order of how much they cost: `ArchiveRoundTripTest`, which fails when a kind is added to the allowlist without
a case proving it can be rebuilt.
- **Send them anyway, marked unverified**, and let the reader see which rows
came with a group signature and which came on one member's word. Cheap, and it
gives up the property that makes the rest of this safe, so it needs its own
screen language rather than a quiet inclusion.
- **Corroborate.** Every member's archive is an independent copy, so a
translation two members' archives agree on is a claim two devices make. That
is a real strengthening and it needs a second archive to compare against,
which the request path already makes ordinary.
- **Sign them.** The group already puts a quorum behind a chapter and its
chunks; putting one behind a translation would make it archivable like
everything else. It is also a product decision about whether translating is an
act of the group or of a member, which is not a decision this document gets to
make.
**The chat is gone and stays gone.** By design, restated here because it is the **The chat is gone and stays gone.** By design, restated here because it is the
first thing a new member will notice and the archive is what makes them expect first thing a new member will notice and the archive is what makes them expect