Skip to content

feat(replay): Add checkpoint save and resume functionality for replays - #2152

Closed
bobtista wants to merge 55 commits into
TheSuperHackers:mainfrom
bobtista:bobtista/replay-checkpoint-save
Closed

feat(replay): Add checkpoint save and resume functionality for replays#2152
bobtista wants to merge 55 commits into
TheSuperHackers:mainfrom
bobtista:bobtista/replay-checkpoint-save

Conversation

@bobtista

@bobtistabobtista commented Jan 19, 2026

Copy link
Copy Markdown

Summary

  • Adds ability to save game state checkpoints during replay playback and resume from them
  • Implements command line options for automated checkpoint save/load (-saveAtFrame, -saveTo, -loadCheckpoint)

New Features:

  • Save checkpoints at any frame during replay playback
  • Resume replay from a saved checkpoint with full state restoration
  • Headless mode support for automated testing

CRC Consistency Fixes:

  • Serialize AI, Pathfinder, AIGroup, and PartitionCell state for checkpoint CRC matching
  • Preload CRC from replay file after checkpoint load for verification
  • Restore RNG state at start of update to ensure deterministic behavior
  • Fix shroud state, weapon timing, and path state corruption after checkpoint load

Testing
-[x] Save checkpoint during replay, load it, verify replay continues without desync
-[x]Test headless mode checkpoint save/load via command line
-[x] Verify CRC matches between original replay and checkpoint-resumed replay

TODO

  • Replicate to Generals

Notes

  • Since game state is saved at the checkpoint moment, any changes that would meaningfully affect game state BEFORE that frame will require a new checkpoint being made. Ideally, fixes are about something that happens after the checkpoint and before the mismatch/crash being fixed.

@greptile-apps

Copy link
Copy Markdown

Greptile Summary

This PR adds checkpoint save/resume functionality for replay debugging, allowing developers to create save points at specific frames during replay playback and resume from those points in headless mode.

Key changes:

  • Extends RecorderClass to implement Snapshot interface for serialization of replay state (filename, file position, CRC queue)
  • Adds CLI options -saveAtFrame, -saveTo, and -loadCheckpoint for checkpoint management
  • Implements continueReplayFromCheckpoint() to load checkpoint and resume replay from saved position
  • Adds headless mode checks in GameState to prevent UI calls and skip visual-only blocks during checkpoint saves
  • Properly handles CRC state restoration by serializing the CRC queue and associated flags

Implementation notes:

  • When loading a checkpoint, the replay filename is restored from the checkpoint itself (not from -replay CLI arg)
  • Visual blocks (CHUNK_TerrainVisual, CHUNK_TacticalView, CHUNK_ParticleSystem, CHUNK_GhostObject) are skipped in headless saves
  • The save/load system handles missing blocks gracefully, making headless checkpoints compatible with the existing save system
  • CRC queue is properly serialized/restored to maintain sync detection after checkpoint resume

Confidence Score: 5/5

  • This PR is safe to merge with proper testing of checkpoint save/load workflow
  • The implementation is well-designed with proper error handling, memory management (CRCInfo cleanup in reset()), and defensive coding. The serialization logic correctly handles all replay state, and the headless mode guards prevent UI crashes. The save/load system's existing tolerance for missing blocks ensures compatibility.
  • No files require special attention. The core implementation in Recorder.cpp and ReplaySimulation.cpp is solid.

Important Files Changed

FilenameOverview
Core/GameEngine/Source/Common/ReplaySimulation.cppImplemented checkpoint save at specified frame and resume functionality with proper error handling
GeneralsMD/Code/GameEngine/Source/Common/Recorder.cppImplemented serialization methods for replay state, added file reopening logic, and proper CRCInfo cleanup in reset()
GeneralsMD/Code/GameEngine/Source/Common/System/SaveGame/GameState.cppAdded CHUNK_Recorder snapshot block, headless mode checks for UI calls, and logic to skip visual blocks in headless saves

Sequence Diagram

sequenceDiagram
participant CLI as Command Line
participant GM as GameMain
participant RS as ReplaySimulation
participant GS as GameState
participant RC as RecorderClass
participant GL as GameLogic
participant FS as FileSystem
Note over CLI,FS: Checkpoint Save Flow
CLI->>GM: -replay file.rep -saveAtFrame 49000 -saveTo checkpoint.sav
GM->>RS: simulateReplays([file.rep])
RS->>RC: playbackFile(file.rep)
RC->>FS: Open replay file
FS-->>RC: File handle
loop Until target frame or end
RS->>GL: UPDATE()
GL-->>RS: Current frame
alt Frame == saveAtFrame
RS->>GS: saveGame(checkpoint.sav)
GS->>RC: xfer(XFER_SAVE)
Note over RC: Saves replay filename,<br/>file position, CRC state
GS->>FS: Write checkpoint file
FS-->>GS: Success
RS->>RC: stopPlayback()
RC->>FS: Close replay file
end
end
Note over CLI,FS: Checkpoint Resume Flow
CLI->>GM: -loadCheckpoint checkpoint.sav
GM->>RS: continueReplayFromCheckpoint(checkpoint.sav)
RS->>GS: loadGame(checkpoint.sav)
GS->>FS: Read checkpoint file
FS-->>GS: Checkpoint data
GS->>RC: xfer(XFER_LOAD)
Note over RC: Restores replay filename,<br/>file position, CRC state
GS->>RC: loadPostProcess()
RC->>FS: reopenReplayFileAtPosition(position)
FS-->>RC: File handle at saved position
loop Until replay end or CRC mismatch
RS->>GL: UPDATE()
GL-->>RS: Current frame
RC->>FS: Read next commands from position
alt CRC mismatch detected
RS-->>GM: Exit with error
end
end
RS-->>GM: Exit code
Loading

@bobtista
bobtista marked this pull request as draft January 19, 2026 21:37
Comment threadGeneralsMD/Code/GameEngine/Source/Common/Recorder.cpp Outdated
Comment on lines +347 to +351
if (TheRecorder->sawCRCMismatch())
{
numErrors++;
break;
}

@Caball009Caball009Jan 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some issues with the CRC computation for the replay from save game.

@helmutbuhler

Copy link
Copy Markdown

Looks nice! Will -loadCheckpoint also work with savegames that the player generated? If so we could check that savegames from retail can still be loaded.

@bobtista
bobtistaforce-pushed the bobtista/replay-checkpoint-save branch 7 times, most recently from 646e04a to 67f736dCompareJanuary 20, 2026 23:42
@bobtista
bobtistaforce-pushed the bobtista/replay-checkpoint-save branch from b6bedd9 to 39210b8CompareJanuary 31, 2026 00:02
bobtista added 18 commits March 9, 2026 14:15
…clearing pending queue and initializing m_lastCell
bobtistaand others added 25 commits March 9, 2026 14:22
The wall pieces loop was using MAX_WALL_PIECES as the array index
instead of i, causing out-of-bounds memory access and potentially
non-deterministic CRC values.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…eckpoint check
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…those with paths
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…s instead of reset
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ckpoint load
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… simulation check
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…l from AI
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…vergence after checkpoint load
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…re object updates
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… determinism
The m_nextDockActionFrame field was not serialized, causing dock actions
to complete at wrong frames after checkpoint load. This led to premature
state transitions and path computations that diverged from baseline.
Co-Authored-By: bobtista <bobtista@users.noreply.github.com>
…e from AI state machine
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ring defeat events after checkpoint load
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… game
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@bobtista
bobtistaforce-pushed the bobtista/replay-checkpoint-save branch from b0138d3 to 6473546CompareMarch 9, 2026 20:32
@github-actions

Copy link
Copy Markdown

⚠️ Title/Commit Validation Failed

Invalid commit messages:

  • Fix TurretStateMachine not serializing base StateMachine state
  • Fix DozerActionStateMachine not serializing base StateMachine state
  • cleanup(replay): Remove debug logging added during checkpoint investigation
  • cleanup(replay): Remove obsolete resetTransientStateForCheckpoint workaround
    PR titles and commit messages must follow conventional commits format:
type: Description
type(scope): Description

Allowed types:bugfix, build, chore, ci, docs, fix, feat, perf, refactor, revert, style, test, tweak, unify

See CONTRIBUTING.md for details.

@bobtista

Copy link
Copy Markdown
Author

Closing this for now - the whackamole process was a grind. There's a fix found along the way though that I'll split out into its own PR.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@bobtista@helmutbuhler@Caball009