fix(dag): sweep freeze window covers the live watcher's actual cadence - #359
Merged
Conversation
The window derived solely from the current persisted config; replan can lower a running node's timeout_ms while the A1/Q2 re-time gate keeps the old watcher on its old cadence, so a config-only window could be shorter than the live watcher's cycle and prematurely settle a healthy node. deadline_ms is only ever written as grant-time + timeout (spawn and each extension; escalations move only the counter), so (deadline - started_at) upper-bounds the watcher's current cadence in every shape. Window = max(config cadence, durable bound). Closes#342
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes#342
What
The sweep's freeze window derived solely from the current persisted config cadence. Replan can lower a running node's
timeout_mswhile the A1/Q2 re-time gate deliberately keeps the old watcher on its old (longer) cadence — a config-only window is then shorter than the live watcher's cycle and sweeps a healthy node mid-decision-window.Fix:
window = max(config cadence, escalateIntervalDurable(deadline_ms, started_at)).deadline_msis only ever written asgrant time + timeout_ms— at spawn and at each deadline extension (escalations move only the counter, never the deadline; verified inprojector.tsNodeTimeoutEscalatedvsNodeDeadlineExtendedfolds). So(deadline − started_at)— the sum of the initial grant plus every extension grant — is always ≥ the last grant, and the last grant's timeout IS the live watcher's cadence. The bound over-estimates after extensions (delays detection, never causes a premature settle); missing columns (legacy rows) return 0 so the config decides alone.Tests
bun run typecheckgreen; new pure tests 2/2Note: the 3 wall-clock-sensitive integration cases in this file fail on this loaded local machine at
f25f37d7etoo (pre-date this change); CI is the arbiter.