perf(state): halve the state, compress it on every core, retry a mutation - #9
Merged
Merged
Conversation
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Measured over 56 batches (runs 25-36): the checkpoint costs 91 s flat per batch, 12% of a run and 21% of a large-file run. The state is 1.04 GB, of which 57% is one column nothing reads. Compression is single-threaded on a 5-core runner. Separately, one transient failure on an upload fails the batch and loses the night, because mutations never retried.
Change
proton_nodes.raw_jsonanddropbox_objects.raw_jsonare written as{}: every field the pipeline reads is its own column, and the inventory already blanked the Dropbox one after the fact (that pass is gone).CHECKPOINTEDbatch items are pruned at plan time (already merged intomirror_objects;CONFIRM_FAILEDrows stay for the operator). Events and commands older than seven days are pruned, keeping the reconcile figures events the report reads from the latest complete walk. Measured on the live state: 1,038 MB to 498 MB, the pushed object 140 MB to 96 MB.lzmareads them as one file, so the object format is unchanged, every history copy stays readable, andtask state-rollbackworks in both directions. Measured 3.2x on 5 cores at identical size. The restore streams instead of holding the whole file in memory.synchronous=NORMAL. WAL mode still survives a process crash; the canonical state is the R2 object fetched at the start of every run, so a runner's disk never outlives what it holds.proton.mutation_max_attempts(default 3) retries a timed-out or failed mutation after the existing backoff, never after an authentication failure. A repeated upload of identical bytes is skipped by the CLI and confirm's accounting still balances, so a retry can only cost time.ProtonCLIErrorcarries the attempt's category.Nothing about what is recorded as mirrored changes. The checkpoint's history-then-canonical ordering and per-batch cadence are unchanged.
Test plan
task test: 160 passed, five new: a multi-stream push restores byte for byte; a transient mutation failure is retried and an auth failure is not; checkpointed items are pruned and failed ones kept; old log rows go and reconcile figures stay; the listing stores no raw JSONtask lint,task checkcheckpoint_secondsinbatches.details_jsonfall from ~91 s andtask statusstill read the state