refactor(chapters): one scaffolding batch for every chapter, not one each
Phase 5 of docs/chapter-uploads.md, and the third fix to something wrong today. AddChapterViewModel was right that translation scaffolding needs its own session -- a chapter's paragraph budget should not be spent on the group's dialects, and how long a chapter may be has nothing to do with how many languages it is read in. It was wrong that it needs one session per chapter, which only becomes visible above one chapter: twelve chapters in a group with three dialects is thirty-six join rows, which fit in a single batch, where the per-chapter shape opens twelve sessions to carry three rows each. Twenty-four proposals where thirteen will do. ChapterScaffolding.propose is now the one answer to what a chapter's scaffolding is, and both view models call it. Two copies of a cross product would be two chances to disagree about what a translation covers. It also fixes a latent bug one chapter could never show. The old code chunked the *translations* by MAX_BATCH_SIZE -- right by coincidence at one chapter, where the cross product and the translation list are the same length, and wrong at any more. 25 chapters in a three-dialect group is 75 rows, which the translation axis would have put in one batch of 75 and proposeSigningBatch would have thrown. Chunking the product is the generalisation, and the test for it is the only one here that fails against the per-chapter shape. Two smaller decisions. The batch takes one timestamp, the latest of the chapters' own, because that is the moment the act finished and the clock would be a third time unrelated to either end of it. And a session whose lead event cannot be read is left out rather than guessed at: a join row naming the wrong chapter is worse than a missing one, because the catch-up can find a missing one. Failure policy unchanged and for the reason it already gave: logged and swallowed, since the chapters are what was asked for. At N the argument is stronger, not weaker -- one cross-product batch is only as available as its worst item. 970 tests, no failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kgothatso Ngako
@@ -0,0 +1,127 @@
|
||||
package press.mantra.compose.managers
|
||||
|
||||
import co.touchlab.kermit.Logger
|
||||
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
||||
import press.mantra.compose.database.model.intermdiate.LocalChatRoom
|
||||
import press.mantra.compose.repository.FrostSigningRepository
|
||||
import press.mantra.compose.repository.MantraRepository
|
||||
|
||||
/**
|
||||
* Asking the group to scaffold newly proposed chapters into every translation
|
||||
* of their version.
|
||||
*
|
||||
* A translation chapter carries no words -- it is `(translation, chapter,
|
||||
* position)` -- and it exists so a translated chunk has somewhere to hang. See
|
||||
* [TranslationScaffold] for the rows themselves; this is the part that reads
|
||||
* the version, finds the chapters as the group will author them, and opens the
|
||||
* sessions.
|
||||
*
|
||||
* **Its own session, not more items on the chapter's.** A chapter is capped at
|
||||
* [TranslationScaffold] -- really at `FrostSigningManager.MAX_BATCH_SIZE` minus
|
||||
* the chapter itself -- and every dialect the group works in would take one of
|
||||
* those places away. How long a chapter may be and how many languages it is
|
||||
* read in have nothing to do with each other, and a shared cap would make the
|
||||
* first move whenever somebody did the second.
|
||||
*
|
||||
* **But one session for all of them, not one per chapter.** This is what the
|
||||
* upload needed and what `AddChapterViewModel` had wrong for a single chapter
|
||||
* only by not noticing: twelve chapters in a group with three dialects is
|
||||
* thirty-six join rows, which fit in one batch, and the per-chapter shape would
|
||||
* have opened twelve sessions to carry three rows each. Twenty-four proposals
|
||||
* where thirteen will do.
|
||||
*
|
||||
* **Failure is logged and swallowed**, as it was before. The chapters have
|
||||
* already been proposed and are the thing that was asked for; losing them
|
||||
* because their scaffolding could not be opened would be the wrong way round. A
|
||||
* join row naming a chapter that never reached a quorum fails a foreign key on
|
||||
* the way in, is logged by `applySignedEvent`, and never becomes a row -- so the
|
||||
* catch-up on the translation is what fills the gap afterwards.
|
||||
*/
|
||||
object ChapterScaffolding {
|
||||
|
||||
private const val TAG = "ChapterScaffolding"
|
||||
|
||||
/**
|
||||
* A chapter that has just been put to the group, named by the session that
|
||||
* will sign it.
|
||||
*
|
||||
* The session rather than the chapter, because there is no chapter yet. Its
|
||||
* id is settled at proposal time -- it is the hash the signers are putting
|
||||
* their shares behind -- and [FrostSigningRepository.unsignedEvents] is how
|
||||
* to read it while the session is still running.
|
||||
*/
|
||||
data class ProposedChapter(
|
||||
val sessionId: String,
|
||||
val index: Int,
|
||||
)
|
||||
|
||||
/**
|
||||
* Opens as few sessions as will carry the join rows for [chapters].
|
||||
*
|
||||
* Chunked over the **cross product**, not over the translations. With one
|
||||
* chapter the two are the same and the old code was right by coincidence;
|
||||
* with twelve, chunking the translations would propose twelve chapters'
|
||||
* worth of rows in one batch per sixty-four dialects, which is the wrong
|
||||
* axis and overflows at the fifth chapter of a three-dialect group.
|
||||
*
|
||||
* Returns the sessions opened, which is nothing the caller has to act on --
|
||||
* it is there so a test can count them.
|
||||
*/
|
||||
suspend fun propose(
|
||||
mantraRepository: MantraRepository,
|
||||
frostSigningRepository: FrostSigningRepository,
|
||||
localChatRoom: LocalChatRoom,
|
||||
userPublicKey: HexKey,
|
||||
artifactVersionId: String,
|
||||
chapters: List<ProposedChapter>,
|
||||
): List<String> {
|
||||
if (chapters.isEmpty()) return emptyList()
|
||||
|
||||
return runCatching {
|
||||
val translations = mantraRepository
|
||||
.getTranslationsForArtifactVersion(artifactVersionId)
|
||||
.ifEmpty { return emptyList() }
|
||||
|
||||
// The chapters as the group will author them. A session whose lead
|
||||
// event cannot be read is left out rather than guessed at: a join
|
||||
// row naming the wrong chapter is worse than one that is missing,
|
||||
// because the catch-up can find a missing one.
|
||||
val sources = chapters.mapNotNull { proposed ->
|
||||
val lead = frostSigningRepository
|
||||
.unsignedEvents(frostSigningRepository.getItems(proposed.sessionId))
|
||||
.firstOrNull()
|
||||
?: return@mapNotNull null
|
||||
|
||||
TranslationScaffold.SourceChapter(id = lead.id, index = proposed.index) to
|
||||
lead.createdAt
|
||||
}
|
||||
if (sources.isEmpty()) return emptyList()
|
||||
|
||||
// One timestamp for the lot, so a scaffolding proposed as one act
|
||||
// reads as one. The latest of the chapters' own, which is the moment
|
||||
// the act finished -- not the clock, which would be a third time
|
||||
// unrelated to either end of it.
|
||||
val createdAt = sources.maxOf { it.second }
|
||||
|
||||
val rows = TranslationScaffold.chaptersOf(
|
||||
translationArtifactVersionIds = translations.map { it.id },
|
||||
chapters = sources.map { it.first },
|
||||
createdAt = createdAt,
|
||||
)
|
||||
|
||||
rows.chunked(FrostSigningManager.MAX_BATCH_SIZE).mapNotNull { batch ->
|
||||
frostSigningRepository.proposeSigningBatch(
|
||||
localChatRoom = localChatRoom,
|
||||
userPublicKey = userPublicKey,
|
||||
events = batch,
|
||||
)?.id
|
||||
}
|
||||
}.onFailure { error ->
|
||||
Logger.withTag(TAG).e(
|
||||
"Proposed ${chapters.size} chapter(s) but could not ask for them in this " +
|
||||
"version's translations",
|
||||
error,
|
||||
)
|
||||
}.getOrDefault(emptyList())
|
||||
}
|
||||
}
|
||||
@@ -19,8 +19,8 @@ import kotlinx.coroutines.launch
|
||||
import press.mantra.compose.database.model.MantraArtifactVersion
|
||||
import press.mantra.compose.database.model.intermdiate.LocalChatRoom
|
||||
import press.mantra.compose.managers.ChapterIndexing
|
||||
import press.mantra.compose.managers.ChapterScaffolding
|
||||
import press.mantra.compose.managers.FrostSigningManager
|
||||
import press.mantra.compose.managers.TranslationScaffold
|
||||
import press.mantra.compose.nostr.nip30303.ChapterEvent
|
||||
import press.mantra.compose.nostr.nip30303.ChapterProposal
|
||||
import press.mantra.compose.nostr.nip30303.ChunkEvent
|
||||
@@ -198,36 +198,18 @@ class AddChapterViewModel(
|
||||
|
||||
/**
|
||||
* Asks the group to scaffold this chapter into every translation of the
|
||||
* version, in sessions of their own.
|
||||
* version, in a session of its own.
|
||||
*
|
||||
* Its own session rather than more items on the chapter's, because the two
|
||||
* would otherwise compete for one batch: a chapter is capped at
|
||||
* [MAX_CHUNKS_PER_CHAPTER] paragraphs, and every dialect the group works in
|
||||
* would take one of those places away. How long a chapter may be and how
|
||||
* many languages it is read in have nothing to do with each other, and a
|
||||
* shared cap would make the first move whenever somebody else did the
|
||||
* second.
|
||||
* The reasoning and the failure policy are [ChapterScaffolding]'s; this is
|
||||
* the one-chapter call into it. Sharing that code with the upload screen is
|
||||
* the point -- there is one answer to what a chapter's scaffolding is, and
|
||||
* two copies of a cross product would be two chances to disagree about what
|
||||
* a translation covers.
|
||||
*
|
||||
* Sessions run independently -- every message carries the session it
|
||||
* belongs to, and each mints its own nonce material -- so a second one
|
||||
* opened here costs the admins another approval and nothing else.
|
||||
*
|
||||
* **What a separate session gives up.** A batch is all-or-nothing; two
|
||||
* sessions are not. If the chapter fails to reach a quorum while this one
|
||||
* succeeds, these are valid signatures over rows naming a chapter nobody
|
||||
* has: they fail a foreign key on the way in, are logged by
|
||||
* `applySignedEvent`, and never become rows. Harmless, and not recoverable
|
||||
* either -- a re-proposed chapter is a different id -- so the catch-up on
|
||||
* the translation is what fills the gap afterwards.
|
||||
*
|
||||
* The chapter is named as the group will author it: [chapterSessionId]'s
|
||||
* first item is the unsigned chapter, whose id is the hash its signers are
|
||||
* putting their shares behind, and it is settled at proposal time rather
|
||||
* than when the quorum arrives.
|
||||
*
|
||||
* Failure here is logged and swallowed. The chapter has already been
|
||||
* proposed and is the thing that was asked for; losing it because its
|
||||
* scaffolding could not be opened would be the wrong way round.
|
||||
* Called before navigating, not after: the route this screen sits on is
|
||||
* popped on success, which clears this view model and takes
|
||||
* `viewModelScope` with it, so work launched afterwards would be cancelled
|
||||
* somewhere in the middle.
|
||||
*/
|
||||
private suspend fun scaffoldIntoTranslations(
|
||||
localChatRoom: LocalChatRoom,
|
||||
@@ -235,40 +217,19 @@ class AddChapterViewModel(
|
||||
chapterIndex: Int,
|
||||
chapterSessionId: String,
|
||||
) {
|
||||
runCatching {
|
||||
val translations = mantraRepository
|
||||
.getTranslationsForArtifactVersion(artifactVersion.id)
|
||||
.ifEmpty { return@runCatching }
|
||||
|
||||
val chapter = frostSigningRepository
|
||||
.unsignedEvents(frostSigningRepository.getItems(chapterSessionId))
|
||||
.firstOrNull() ?: return@runCatching
|
||||
|
||||
// One session per MAX_BATCH_SIZE translations. A group with more
|
||||
// dialects than a batch holds is not a group this should refuse;
|
||||
// it is one that answers twice.
|
||||
translations.chunked(FrostSigningManager.MAX_BATCH_SIZE).forEach { batch ->
|
||||
frostSigningRepository.proposeSigningBatch(
|
||||
localChatRoom = localChatRoom,
|
||||
userPublicKey = activeUserPublicKey,
|
||||
events = TranslationScaffold.chaptersOf(
|
||||
translationArtifactVersionIds = batch.map { it.id },
|
||||
chapters = listOf(
|
||||
TranslationScaffold.SourceChapter(
|
||||
id = chapter.id,
|
||||
index = chapterIndex,
|
||||
)
|
||||
),
|
||||
createdAt = chapter.createdAt,
|
||||
)
|
||||
ChapterScaffolding.propose(
|
||||
mantraRepository = mantraRepository,
|
||||
frostSigningRepository = frostSigningRepository,
|
||||
localChatRoom = localChatRoom,
|
||||
userPublicKey = activeUserPublicKey,
|
||||
artifactVersionId = artifactVersion.id,
|
||||
chapters = listOf(
|
||||
ChapterScaffolding.ProposedChapter(
|
||||
sessionId = chapterSessionId,
|
||||
index = chapterIndex,
|
||||
)
|
||||
}
|
||||
}.onFailure { error ->
|
||||
logger.e(
|
||||
"Proposed the chapter but could not ask for it in this version's translations",
|
||||
error
|
||||
)
|
||||
}
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
companion object {
|
||||
|
||||
@@ -18,6 +18,7 @@ import press.mantra.compose.database.model.MantraArtifactVersion
|
||||
import press.mantra.compose.database.model.intermdiate.LocalChatRoom
|
||||
import press.mantra.compose.files.PickedFile
|
||||
import press.mantra.compose.managers.ChapterIndexing
|
||||
import press.mantra.compose.managers.ChapterScaffolding
|
||||
import press.mantra.compose.managers.FrostSigningManager
|
||||
import press.mantra.compose.nostr.nip30303.ChapterEvent
|
||||
import press.mantra.compose.nostr.nip30303.ChunkEvent
|
||||
@@ -208,6 +209,7 @@ class UploadChaptersViewModel(
|
||||
var index = nextIndex(artifactVersion)
|
||||
var proposed = 0
|
||||
var failed = 0
|
||||
val scaffolded = mutableListOf<ChapterScaffolding.ProposedChapter>()
|
||||
|
||||
for (id in queue) {
|
||||
val row = (_uiState.value as? UploadChaptersUIState.Loaded)
|
||||
@@ -239,6 +241,10 @@ class UploadChaptersViewModel(
|
||||
|
||||
if (session != null) {
|
||||
markProposed(id, session.id)
|
||||
scaffolded += ChapterScaffolding.ProposedChapter(
|
||||
sessionId = session.id,
|
||||
index = index,
|
||||
)
|
||||
proposed++
|
||||
index++
|
||||
} else {
|
||||
@@ -247,6 +253,23 @@ class UploadChaptersViewModel(
|
||||
}
|
||||
}
|
||||
|
||||
// One batch for every chapter that went, not one per chapter. Twelve
|
||||
// chapters in a three-dialect group is thirty-six join rows, which
|
||||
// fit in a single session; the per-chapter shape would have opened
|
||||
// twelve of them to carry three rows each.
|
||||
//
|
||||
// After the loop rather than inside it, so a chapter that failed to
|
||||
// propose contributes no scaffolding and the ones that succeeded
|
||||
// still get theirs.
|
||||
ChapterScaffolding.propose(
|
||||
mantraRepository = mantraRepository,
|
||||
frostSigningRepository = frostSigningRepository,
|
||||
localChatRoom = localChatRoom,
|
||||
userPublicKey = activeUserPublicKey,
|
||||
artifactVersionId = artifactVersion.id,
|
||||
chapters = scaffolded,
|
||||
)
|
||||
|
||||
passes++
|
||||
updateLoaded {
|
||||
it.copy(
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
package press.mantra.compose.managers
|
||||
|
||||
import androidx.room3.Room
|
||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||
import fr.acinq.bitcoin.ByteVector32
|
||||
import fr.acinq.bitcoin.PrivateKey
|
||||
import fr.acinq.bitcoin.crypto.frost.Frost
|
||||
import fr.acinq.bitcoin.crypto.frost.KeyMaterial
|
||||
import kotlin.test.AfterTest
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
import kotlin.test.assertTrue
|
||||
import kotlinx.coroutines.CoroutineScope
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.SupervisorJob
|
||||
import kotlinx.coroutines.runBlocking
|
||||
import press.mantra.compose.database.MantraDatabase
|
||||
import press.mantra.compose.database.builder.getRoomDatabase
|
||||
import press.mantra.compose.database.model.ChatRoom
|
||||
import press.mantra.compose.database.model.DkgParticipantMessage
|
||||
import press.mantra.compose.database.model.DkgSession
|
||||
import press.mantra.compose.database.model.MantraArtifact
|
||||
import press.mantra.compose.database.model.MantraArtifactVersion
|
||||
import press.mantra.compose.database.model.MantraDialect
|
||||
import press.mantra.compose.database.model.MantraTranslationArtifactVersion
|
||||
import press.mantra.compose.database.model.NostrEvent
|
||||
import press.mantra.compose.database.model.Profile
|
||||
import press.mantra.compose.database.model.intermdiate.LocalChatRoom
|
||||
import press.mantra.compose.database.model.types.DkgRitualStage
|
||||
import press.mantra.compose.database.repository.DatabaseFrostSigningRepository
|
||||
import press.mantra.compose.database.repository.DatabaseMantraRepository
|
||||
import press.mantra.compose.extensions.toHex
|
||||
import press.mantra.compose.nostr.dkg.DkgRitualEvents
|
||||
import press.mantra.compose.nostr.nip30303.ChapterEvent
|
||||
import press.mantra.compose.nostr.nip30303.ChunkEvent
|
||||
import press.mantra.compose.nostr.nip30303.TranslationChapterEvent
|
||||
import press.mantra.compose.repository.FrostSigningRepository
|
||||
import press.mantra.compose.repository.MantraRepository
|
||||
import press.mantra.compose.text.Markdown
|
||||
|
||||
/**
|
||||
* How many sessions a group is asked for when chapters arrive together.
|
||||
*
|
||||
* The scaffolding has to be its own session -- a chapter's paragraph budget
|
||||
* should not be spent on the group's dialects -- but it does not have to be one
|
||||
* session *per chapter*, and that only becomes visible above one chapter. The
|
||||
* first test here is the arithmetic the upload rests on: twelve chapters and
|
||||
* three dialects is thirty-six join rows, which is one batch rather than
|
||||
* twelve.
|
||||
*/
|
||||
class ChapterScaffoldingJvmTest {
|
||||
private val participants = 3
|
||||
private val threshold = 2
|
||||
|
||||
private val keyMaterial: KeyMaterial = Frost.trustedDealerKeygen(
|
||||
thresholdSecretKey = PrivateKey(
|
||||
ByteVector32("1c0ffee0000000000000000000000000000000000000000000000000000000a1")
|
||||
),
|
||||
nParticipants = participants,
|
||||
threshold = threshold,
|
||||
)
|
||||
|
||||
private val thresholdPublicKey = keyMaterial.thresholdPublicKey.value.toHex()
|
||||
private val roomId = SharedKeyDerivation.marmotGroupId(thresholdPublicKey)
|
||||
|
||||
private val ceremonyId = "ceremony".padEnd(64, '0')
|
||||
private val members = listOf("a", "b", "c").map { it.repeat(64) }
|
||||
private val hostKeys = listOf("2a", "2b", "2c").map { it.padEnd(66, '0') }
|
||||
|
||||
private val proposer = members.first()
|
||||
private val dialectId = "d".repeat(64)
|
||||
private val artifactId = "a1".padEnd(64, '0')
|
||||
private val artifactVersionId = "b".repeat(64)
|
||||
|
||||
private lateinit var db: MantraDatabase
|
||||
private lateinit var room: LocalChatRoom
|
||||
private lateinit var mantraRepository: MantraRepository
|
||||
private lateinit var frostSigningRepository: FrostSigningRepository
|
||||
|
||||
@AfterTest
|
||||
fun closeDatabase() {
|
||||
if (::db.isInitialized) db.close()
|
||||
}
|
||||
|
||||
private suspend fun openDevice() {
|
||||
db = getRoomDatabase(Room.inMemoryDatabaseBuilder<MantraDatabase>())
|
||||
val scope = CoroutineScope(Dispatchers.Default + SupervisorJob())
|
||||
mantraRepository = DatabaseMantraRepository(db, scope)
|
||||
frostSigningRepository = DatabaseFrostSigningRepository(db, scope)
|
||||
|
||||
val nostrEventId = "e$proposer".take(64)
|
||||
db.nostrEventDao().upsert(
|
||||
NostrEvent(
|
||||
id = nostrEventId,
|
||||
pubKey = proposer,
|
||||
kind = 0,
|
||||
tags = emptyArray(),
|
||||
content = "{}",
|
||||
sig = "0".repeat(128),
|
||||
)
|
||||
)
|
||||
db.profileDao().upsert(Profile(publicKey = proposer, nostrEventId = nostrEventId))
|
||||
|
||||
val chatRoom = ChatRoom(
|
||||
id = roomId,
|
||||
userPublicKey = proposer,
|
||||
subject = "#Ekklesia",
|
||||
description = SharedKeyDerivation.describe("Admins of the group."),
|
||||
mlsGroupState = null,
|
||||
)
|
||||
db.chatRoomDao().upsert(chatRoom)
|
||||
|
||||
db.dkgSessionDao().upsert(
|
||||
DkgSession(
|
||||
id = ceremonyId,
|
||||
chatRoomId = roomId,
|
||||
coordinatorPublicKey = proposer,
|
||||
userPublicKey = proposer,
|
||||
threshold = threshold,
|
||||
participantCount = participants,
|
||||
stage = DkgRitualStage.COMPLETE,
|
||||
hostPublicKey = hostKeys[0],
|
||||
round1Random = "1".repeat(64),
|
||||
round2AuxRandom = "2".repeat(64),
|
||||
thresholdPublicKey = thresholdPublicKey,
|
||||
secretShare = keyMaterial.secretShares[0].value.toHex(),
|
||||
publicShares = keyMaterial.publicShares.joinToString(",") { it.value.toHex() },
|
||||
)
|
||||
)
|
||||
members.forEachIndexed { index, member ->
|
||||
db.dkgSessionDao().upsert(
|
||||
DkgParticipantMessage(
|
||||
sessionId = ceremonyId,
|
||||
participantPublicKey = member,
|
||||
kind = DkgRitualEvents.HOST_KEY,
|
||||
payload = hostKeys[index],
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
db.mantraDialectDao().upsert(
|
||||
MantraDialect(
|
||||
id = dialectId,
|
||||
publicKey = roomId,
|
||||
name = "Setswana",
|
||||
country = "BW",
|
||||
language = "tn",
|
||||
signature = "",
|
||||
chatRoomId = roomId,
|
||||
)
|
||||
)
|
||||
db.mantraArtifactDao().upsert(
|
||||
MantraArtifact(
|
||||
id = artifactId,
|
||||
publicKey = roomId,
|
||||
name = "To Kill a Mockingbird",
|
||||
url = "https://example.test/mockingbird",
|
||||
visibility = "private",
|
||||
dialectId = dialectId,
|
||||
license = "cc",
|
||||
chatRoomId = roomId,
|
||||
signature = "",
|
||||
)
|
||||
)
|
||||
db.mantraArtifactVersionDao().upsert(
|
||||
MantraArtifactVersion(
|
||||
id = artifactVersionId,
|
||||
artifactId = artifactId,
|
||||
publicKey = roomId,
|
||||
versionLabel = "1.0",
|
||||
chatRoomId = roomId,
|
||||
signature = "",
|
||||
)
|
||||
)
|
||||
|
||||
room = LocalChatRoom(chatRoom = chatRoom)
|
||||
}
|
||||
|
||||
private suspend fun addTranslations(count: Int) {
|
||||
(0 until count).forEach { ordinal ->
|
||||
db.mantraTranslationArtifactVersionDao().upsert(
|
||||
MantraTranslationArtifactVersion(
|
||||
id = "translation$ordinal".padEnd(64, '0'),
|
||||
publicKey = roomId,
|
||||
artifactVersionId = artifactVersionId,
|
||||
dialectId = dialectId,
|
||||
name = "Translation $ordinal",
|
||||
visibility = "private",
|
||||
license = "cc",
|
||||
signature = "",
|
||||
chatRoomId = roomId,
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/** Proposes [count] chapters the way the upload screen does, in index order. */
|
||||
private suspend fun proposeChapters(count: Int): List<ChapterScaffolding.ProposedChapter> =
|
||||
(0 until count).map { index ->
|
||||
val originalText = "Paragraph one.\n\nParagraph two."
|
||||
val session = FrostSigningManager.proposeSigningBatch(
|
||||
database = db,
|
||||
localChatRoom = room,
|
||||
userPublicKey = proposer,
|
||||
lead = ChapterEvent.build(
|
||||
artifactVersionId = artifactVersionId,
|
||||
name = "Chapter $index",
|
||||
originalText = originalText,
|
||||
index = index,
|
||||
wordCount = Markdown.wordCount(originalText),
|
||||
characterCount = Markdown.characterCount(originalText),
|
||||
),
|
||||
dependents = ChunkEvent::splitOf,
|
||||
)
|
||||
ChapterScaffolding.ProposedChapter(sessionId = session.id, index = index)
|
||||
}
|
||||
|
||||
private suspend fun scaffold(chapters: List<ChapterScaffolding.ProposedChapter>) =
|
||||
ChapterScaffolding.propose(
|
||||
mantraRepository = mantraRepository,
|
||||
frostSigningRepository = frostSigningRepository,
|
||||
localChatRoom = room,
|
||||
userPublicKey = proposer,
|
||||
artifactVersionId = artifactVersionId,
|
||||
chapters = chapters,
|
||||
)
|
||||
|
||||
private suspend fun itemEvents(sessionId: String): List<Event> =
|
||||
db.frostSigningSessionDao().getItems(sessionId).map { Event.fromJson(it.unsignedEventJson) }
|
||||
|
||||
@Test
|
||||
fun `twelve chapters and three dialects is one scaffolding session`() = runBlocking {
|
||||
openDevice()
|
||||
addTranslations(3)
|
||||
|
||||
val sessions = scaffold(proposeChapters(12))
|
||||
|
||||
// The arithmetic the upload rests on. Thirty-six join rows fit in one
|
||||
// batch; a session per chapter would have been twelve of them carrying
|
||||
// three rows each.
|
||||
assertEquals(1, sessions.size)
|
||||
assertEquals(36, itemEvents(sessions.single()).size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `every row is a translation chapter naming a chapter the group will author`() =
|
||||
runBlocking {
|
||||
openDevice()
|
||||
addTranslations(2)
|
||||
|
||||
val chapters = proposeChapters(2)
|
||||
val sessions = scaffold(chapters)
|
||||
val rows = itemEvents(sessions.single())
|
||||
|
||||
assertTrue(rows.all { it.kind == TranslationChapterEvent.KIND })
|
||||
|
||||
// The ids come from the chapters' own sessions, which is the only
|
||||
// place they exist before a quorum signs: a hash over the group's
|
||||
// key at the room's path.
|
||||
val chapterIds = chapters.map { proposed ->
|
||||
itemEvents(proposed.sessionId).first().id
|
||||
}
|
||||
val named = rows.map { row ->
|
||||
TranslationChapterEvent(
|
||||
row.id, row.pubKey, row.createdAt, row.tags, row.content, row.sig
|
||||
).chapterId()
|
||||
}
|
||||
|
||||
assertEquals(chapterIds.toSet(), named.toSet())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a chapter keeps its own position, not the list's`() = runBlocking {
|
||||
openDevice()
|
||||
addTranslations(1)
|
||||
|
||||
// A catch-up proposes a subset, so counting the list would renumber the
|
||||
// chapters it happens to be filling in.
|
||||
val sessions = scaffold(
|
||||
listOf(
|
||||
ChapterScaffolding.ProposedChapter(
|
||||
sessionId = proposeChapters(1).single().sessionId,
|
||||
index = 7,
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
val row = itemEvents(sessions.single()).single()
|
||||
assertEquals(
|
||||
7,
|
||||
TranslationChapterEvent(
|
||||
row.id, row.pubKey, row.createdAt, row.tags, row.content, row.sig
|
||||
).index(),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `the cross product is what gets chunked, not the translations`() = runBlocking {
|
||||
openDevice()
|
||||
addTranslations(3)
|
||||
|
||||
// 25 chapters x 3 dialects is 75 rows, which is two batches. Chunking
|
||||
// the translations instead would have put all 75 in one and thrown --
|
||||
// which is the bug this generalisation fixes, and it only appears above
|
||||
// one chapter.
|
||||
val sessions = scaffold(proposeChapters(25))
|
||||
|
||||
assertEquals(2, sessions.size)
|
||||
assertEquals(
|
||||
FrostSigningManager.MAX_BATCH_SIZE,
|
||||
itemEvents(sessions.first()).size,
|
||||
)
|
||||
assertEquals(75 - FrostSigningManager.MAX_BATCH_SIZE, itemEvents(sessions.last()).size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a version with no translations is asked for nothing`() = runBlocking {
|
||||
openDevice()
|
||||
|
||||
assertEquals(emptyList(), scaffold(proposeChapters(3)))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `no chapters is asked for nothing`() = runBlocking {
|
||||
openDevice()
|
||||
addTranslations(3)
|
||||
|
||||
assertEquals(emptyList(), scaffold(emptyList()))
|
||||
}
|
||||
}
|
||||
@@ -557,14 +557,32 @@ during an upload.
|
||||
`TranslationScaffold.chaptersOf(translationArtifactVersionIds, chapters,
|
||||
createdAt)` already takes a list and already builds the full cross product in
|
||||
translation-then-reading order. So lift the scaffolding out of the per-chapter
|
||||
loop: collect each chapter's `SourceChapter(id, index)` from its session's
|
||||
unsigned lead event, and after the last chapter is proposed, propose the whole
|
||||
cross product in `chunked(MAX_BATCH_SIZE)` batches — which is what `addChapter`
|
||||
already does across *translations* and what
|
||||
`TranslationArtifactVersionDetailViewModel` does across chapters.
|
||||
loop into `ChapterScaffolding.propose`, which both view models now call:
|
||||
collect each chapter's `SourceChapter(id, index)` from its session's unsigned
|
||||
lead event, and after the last chapter is proposed, propose the whole cross
|
||||
product in `chunked(MAX_BATCH_SIZE)` batches.
|
||||
|
||||
Twelve chapters × three dialects: from 12 scaffolding sessions to 1.
|
||||
|
||||
**And it fixes a latent bug that one chapter could never show.** The old code
|
||||
chunked the *translations* — right by coincidence at one chapter, where the
|
||||
cross product and the translation list are the same length, and wrong at any
|
||||
more: 25 chapters in a three-dialect group is 75 rows, which the translation
|
||||
axis would have put in a single batch of 75 and `proposeSigningBatch` would have
|
||||
thrown. Chunking the product is the generalisation, and the test for it is the
|
||||
only one here that would fail against the per-chapter shape.
|
||||
|
||||
Two smaller decisions the build had to make:
|
||||
|
||||
- **One timestamp for the batch, the latest of the chapters' own.**
|
||||
`chaptersOf` stamps every row with one `createdAt` so a scaffolding proposed
|
||||
as one act reads as one; with N chapters there are N candidates. The latest is
|
||||
the moment the act finished. Reading the clock instead would be a third time
|
||||
unrelated to either end of it.
|
||||
- **A session whose lead event cannot be read is left out, not guessed at.** A
|
||||
join row naming the wrong chapter is worse than a missing one, because the
|
||||
catch-up can find a missing one.
|
||||
|
||||
Keep the existing failure policy verbatim, including its reasoning: scaffolding
|
||||
failure is logged and swallowed, because the chapters are the thing that was
|
||||
asked for, and a scaffolding row naming a chapter that never reached quorum
|
||||
@@ -575,7 +593,15 @@ only as available as its worst item, and some of those chapters may genuinely
|
||||
fail.
|
||||
|
||||
`AddChapterViewModel` keeps its one-chapter call into the same helper, so there
|
||||
is one piece of code deciding what a chapter's scaffolding is.
|
||||
is one piece of code deciding what a chapter's scaffolding is. The upload calls
|
||||
it once, after its loop, with only the chapters that actually went — so a
|
||||
chapter that failed to propose contributes no scaffolding and the ones that
|
||||
succeeded still get theirs.
|
||||
|
||||
**Tests:** `ChapterScaffoldingJvmTest`, six of them, over a real database and
|
||||
real proposals. The arithmetic (12 × 3 = one session of 36 rows), the ids coming
|
||||
from the chapters' own sessions, a chapter keeping its own index rather than the
|
||||
list's, the cross-product chunking at 25 × 3 = 75, and the two empty cases.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user