refresh stale stats across inode rotation (Phase 3) - #736
Merged
Merged
Conversation
The May-1 fix that stopped flagging every comic modified when inodes rotate (689bbc4) correctly silenced the false-modify storm but left DB stat rows permanently lying about disk reality afterward — once a Docker bind-mount remount rotated the kernel's inode space, the DB's stored inodes no longer pointed at anything on disk, and ``_find_moved_paths`` had a stale lookup table from there on. The ``_is_move_compatible`` guard added in the previous PR suppresses the corruption that produced, but legitimate cross-remount renames still degrade to delete+add (and the user loses bookmarks on the recreated comic.pk). Phase 3 closes the loop. ``SnapshotDiff`` now emits ``StaleStatRefresh`` payloads for every ``data.unchanged`` path whose mtime+size still match but whose inode has rotated. The poller writes those fresh stats back to the appropriate model with one ``bulk_update`` per affected table, with ``fields=["stat"]`` only — ``updated_at`` is intentionally NOT bumped, since the file's content is unchanged and bookmark "fresh" semantics must hold. The refresh is skipped on ``force=True`` polls because force routes every path through the import pipeline (where ``presave`` already rewrites stats), and we don't want to double-write. To target the right model with one bulk write per table, ``DatabaseSnapshot`` now tracks the source model for each path in a new ``_path_to_model`` map; ``Snapshot.model_for_path`` exposes it, and ``DiskSnapshot`` leaves the map empty. Operational characteristics: - Zero DB writes when inodes are stable (the common case). The diff loop runs in O(unchanged) time but the per-path inode comparison is just a tuple equality. - One ``bulk_update`` per affected model on a remount (Comic, Folder, FailedImport, CustomCover separately). - Idempotent: a second cycle after a successful refresh sees inodes matching disk and emits zero refreshes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the loop that #735 left open: refresh DB stats when an inode rotates without content changing, so the diff's lookup keyspace stays synchronized with disk reality across Docker bind-mount remounts.
Why
The May 1 fix that stopped flagging every comic modified when inodes rotate (689bbc4) correctly silenced the false-modify storm Docker remounts produced — but it left DB stat rows permanently lying about disk reality afterward. Once a remount rotated the kernel's inode space, every
_find_moved_pathscall ran against a lookup table that no longer matched any of the inodes the disk would emit.#735's
_is_move_compatibleguard suppresses the corruption that mismatch produced (file rows whosepathwas rewritten to a directory). But suppression isn't repair: legitimate cross-remount renames still degrade to delete+add, and the user loses bookmarks tied to the recreatedcomic.pk.This PR refreshes the DB stat in place —
bulk_update(fields=["stat"])only,updated_atis not bumped because the file's content hasn't changed.Trigger condition
A path P qualifies for refresh when:
data.unchanged(present in both DB and disk snapshots)data.modified(mtime and size match disk)db_snap.inode(P) != disk_snap.inode(P)(inode rotated under us)That predicate fingerprints the Docker remount case and the rsync-with-timestamps file-replacement case. It produces zero refreshes when inodes are stable, so the common case has no DB writes.
Where in the cycle
_queue_poll_events, between_get_diffand the import-task build:Skipped on
force=Truepolls because force routes every path through the import pipeline (wherepresave()already rewrites stats), and double-writing serves no purpose.Implementation notes
DatabaseSnapshotnow tracks the source model for each path in a new_path_to_modelmap.Snapshot.model_for_path()exposes it;DiskSnapshotleaves it empty.SnapshotDiffexposesstale_stat_refreshes: tuple[StaleStatRefresh, ...]. Each carriespath,model, and the fresh diskos.stat_result.bulk_updateper cycle.Operational
bulk_updateper affected model on a remount.Test plan
make fixandmake lintclean (preexistingremarkexit-1 unrelated)pytest tests/— 48 passed (5 new diff-layer unit tests, 2 new Django integration tests, 41 pre-existing)updated_aton those rows is unchanged--forcepoll: confirm the refresh path is skipped (force path handles its own stat updates viapresave)🤖 Generated with Claude Code