Uh oh!
There was an error while loading. Please reload this page.
fix(fleet): don't kill a healthy --web-bg run on a run-record write failure - #440
Merged
Jason Robert (jrob5756) merged 3 commits intoAug 14, 2026
Merged
Conversation
…ailure The launch health gate (Fleet Manager D2) polled for the child's run record and terminated the child if it wasn't found within the timeout, even when the child was alive and its dashboard was reachable. This downgrades that to a warning (BackgroundLaunch.run_record_written=False) surfaced via a note pointing at the captured stderr log and a Fleet Manager TUI notification; only an actually-dead child, or one whose dashboard has gone unreachable, still fails the launch. Also introduces conductor.run_id as the single definition of the run_id format, replacing divergent patterns in fleet/records.py and engine/event_log.py (the latter lowercased its input), which could cause a resumed --web-bg run's launch-gate poll to look for the wrong key and kill a successful resume. Closes#435
…gate - fleet/records.py: fix _LOG_STEM_TIMESTAMP_RE regression -- the widened run-id charset can span hyphens, so a plain re.search anchored on the FIRST timestamp-shaped segment in a log's filename instead of the last one whenever a workflow name itself contained a YYYYMMDD-HHMMSS-shaped segment. Anchor with a greedy `.*` prefix + `.match()` (mirroring fleet/history.py's parser) and correct the comment that asserted the opposite of the truth. Adds a regression test. - cli/bg_runner.py: add stage-two coverage for `run_record_written` through the STARTED / TIMED_OUT / CHILD_EXITED probes (previously only exercised via the `start_timeout == 0` early return), closing the gap where the STARTED arm could be hard-coded True with the full suite green. - cli/app.py / fleet/tui/screens/new_run.py: add coverage for the resume --web-bg "could not register itself for discovery" notice and the New Run screen's warning notification, both previously untested (the latter was structurally unreachable behind a `Mock()` with an auto-truthy `.run_record_written`). - Recommendations applied: re-read the run record once more after the 1s reachability re-probe succeeds before declaring the discovery record missing; drop the redundant stderr tail from the downgrade warning; restore the missing Raises: bullet and refresh the stale "two-stage"/PID-file module docstring in bg_runner.py; tighten run_id.RUN_ID_PATTERN_SOURCE's upper bound from 200 to 64 chars (a 200-char CONDUCTOR_RUN_ID could blow past a filesystem's per-component name limit and abort the workflow instead of falling back to a fresh id); anchor fleet/retention.py's companion-log glob so one run's id being a hyphen-boundary suffix of another's can't sweep up a foreign run's captured logs; rename run_id.RUN_ID_RE to the internal _RUN_ID_RE and drop its redundant \A...\Z anchors (fullmatch already anchors); fix a pre-existing environment-dependent test flake (test_filenames_unique_for_simultaneous_starts assumed no ambient CONDUCTOR_RUN_ID). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Jason Robert (jrob5756)
marked this pull request as ready for review
August 14, 2026 17:27
…d-launch-gate # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--web-bg/resume --web-bgchild and fail the launch if it couldn't confirm the child's run record within the poll timeout, even when the child was alive and its dashboard was reachable — treating a bookkeeping failure as a workflow failure.BackgroundLaunch.run_record_written=False, surfaced via a note pointing at the captured stderr log and a Fleet Manager TUI notification. Only a child that is actually dead, or whose dashboard has gone unreachable, still fails the launch.conductor.run_idas the single definition of therun_idformat. Previouslyfleet/records.pyenforced a broad path-safe pattern whileengine/event_log.pyindependently enforced a narrower hex-only pattern and lowercased its input — a resumed--web-bgrun whose checkpointrun_idcontained uppercase characters could be silently folded to a different value, causing the parent's launch-gate poll to look for a key the child never wrote and kill the resumed run 15 seconds after a successful start.Closes#435
Test plan
uv run pytest tests/test_cli/test_bg_runner.py tests/test_engine/test_event_log.py tests/test_fleet/