harden: lock episodic writes + fix pi install.sh silent orphan - #19
Merged
Conversation
Two pre-existing infrastructure bugs flagged during the PR #17 cross-model review, fixed here against master because they predate that PR and affect every harness. ## 1. Concurrent writes to AGENT_LEARNINGS.jsonl post_execution.py and on_failure.py both did plain `open(EPISODIC, "a")` → `f.write(json.dumps(entry) + "\n")`. POSIX O_APPEND makes single `write(2)` calls atomic only up to PIPE_BUF (4 KB on Linux/macOS). In practice most entries stay under that ceiling and the unlocked code never corrupts, but the `reflection` field is uncapped in log_execution and can easily exceed 4 KB on high-importance failure logs. Every downstream reader (auto_dream.py, cluster.py, context_budget.py, show.py) skips `json.JSONDecodeError` lines silently — so one over-PIPE_BUF interleave = one episodic entry gone with no signal. Fix: new `_episodic_io.append_jsonl()` helper that opens in append- binary mode (no Python text-mode buffering quirks) and wraps the write in `fcntl.flock(LOCK_EX)`. Shared by both writers. On platforms without fcntl (native Windows Python) behavior falls back to the pre-fix unlocked append; WSL, git-bash/Cygwin, macOS, Linux all have fcntl. Verified: 40 concurrent writers × 500 entries × 2 KB reflection each → 20,000 parseable lines, zero corruption. ## 2. pi install.sh silently leaves stale skills on re-install `ln -sfn src dest` where `dest` is a REAL directory (e.g. from an earlier copy-fallback install) silently creates `dest/<basename-of-src>` INSIDE the dir and exits 0. The existing `if ln -sfn; then` branch took the success path, the `rm -rf + cp` fallback never ran, and orphans stuck around forever. Verified on macOS, confirmed the symlink-inside-dir behavior. Fix: check `-L` (symlink) and `-d` (real dir) explicitly before calling `ln -sfn`, mirror the pattern used by the codex adapter (PR #16 follow-up). Existing symlink → cheap repoint. Real directory → rsync --delete when available, rm+cp otherwise. Non-existent → symlink or copy fallback. Same three-branch shape, no more silent wrong behavior. Verified: re-install after orphan-skill was added to a real-dir `.pi/skills` → rsync --delete removes the orphan.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 90d77d9e38
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| os.makedirs(os.path.dirname(path), exist_ok=True) | ||
| with open(path, "ab") as f: | ||
| if _HAVE_FLOCK: | ||
| fcntl.flock(f.fileno(), fcntl.LOCK_EX) |
There was a problem hiding this comment.
append_jsonl only falls back when fcntl is missing, but fcntl.flock(...) can still raise OSError on filesystems that do not support advisory locks (for example, some network or virtual mounts). In that case both log_execution and on_failure will raise and can abort the hook path instead of just recording without locking, which is a regression from the previous append-only behavior.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two shared-infrastructure bugs, separate from PRs #16 and #17 because they predate both.
Summary
1. AGENT_LEARNINGS.jsonl concurrent writes —
post_execution.pyandon_failure.pyboth do plainopen("a")+f.write(). POSIX O_APPEND is atomic only up to PIPE_BUF (4 KB on Linux/macOS). Thereflectionfield inlog_executionis uncapped and can exceed 4 KB on high-importance failure logs. Every downstream reader (auto_dream.py,cluster.py,context_budget.py,show.py) skipsJSONDecodeErrorlines silently, so one over-PIPE_BUF interleave = one episodic entry gone with no signal. New_episodic_io.append_jsonl()helper: append-binary open +fcntl.flock(LOCK_EX). Cross-process mutex, no new deps. Fall back to unlocked append on platforms without fcntl (native Windows Python; WSL/git-bash/macOS/Linux all have it).2. pi install.sh silently leaves stale skills on re-install —
ln -sfn src destwheredestis a real directory (from a prior copy-fallback install) silently createsdest/<basename-of-src>INSIDE the dir and exits 0. The oldif ln -sfn; thenbranch took the success path, therm -rf + cpfallback never ran, orphans stuck around forever. Confirmed on macOS. Fixed by checking-L/-dexplicitly beforeln -sfn, mirroring the pattern I used in the codex adapter (PR #16 follow-up).Test plan
.pi/skillswith an injected orphan skill → orphan deleted via rsync --delete (rm+cp fallback if rsync unavailable)Scope notes
JSONDecodeErrortolerance stays, but with the lock there should be no corrupted lines for them to skip.