Skip to content

fix(io): force UTF-8 on stdout/stderr and candidate writes - #61

Merged
codejunkie99 merged 1 commit into
codejunkie99:masterfrom
diazMelgarejo:atomic-02-utf8-encoding-fixes
Aug 6, 2026
Merged

fix(io): force UTF-8 on stdout/stderr and candidate writes#61
codejunkie99 merged 1 commit into
codejunkie99:masterfrom
diazMelgarejo:atomic-02-utf8-encoding-fixes

Conversation

@diazMelgarejo

@diazMelgarejo diazMelgarejo commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Atomic PR 2 of 3 (PR 1, episodic-mirror, merged as #57).

Summary

Two small UTF-8 robustness fixes in .agent/tools/learn.py, both stdlib-only:

  1. Force UTF-8 on sys.stdout/sys.stderr at import time (guarded by hasattr(..., "reconfigure") for older Python), so non-ASCII claim text prints correctly regardless of the host locale — matters most on Windows, where the default console codepage isn't UTF-8.
  2. open(path, "w")open(path, "w", encoding="utf-8") for the candidate JSON write, so it doesn't silently pick up the platform default encoding.

Rebased onto current master — the branch originally also carried PR #1's commits (stacked, per the original 3-PR plan); those are now dropped since #57 already merged that content, leaving only this PR's own diff.

Tests

Full suite passes on top of current master: 177 passed (+ 3 in .agent/tools/test_learn_episodic_mirror.py), 1 pre-existing unrelated failure (test_unknown_executable_is_a_structured_start_failure, a macOS PermissionError-vs-FileNotFoundError platform quirk unrelated to this change).

Note

Force UTF-8 encoding on stdout/stderr and candidate file writes in learn.py

  • Reconfigures sys.stdout and sys.stderr to use UTF-8 with errors='replace' at module startup in learn.py, replacing unencodable characters instead of raising errors.
  • The stage() function now explicitly passes encoding='utf-8' when writing candidate JSON files, replacing reliance on the platform default encoding.

Macroscope summarized 5e38f01.

Two related Windows-compatibility fixes:

- Reconfigure stdout/stderr to UTF-8 (errors=replace) at import, so a lesson
  claim containing non-ASCII (arrows, em-dashes, accented text) doesn't raise
  UnicodeEncodeError under a cp1252 console on Windows.
- Write the candidate JSON file with explicit encoding="utf-8" rather than
  the platform default, so candidates round-trip identically across OSes.

Both are additive and platform-safe (the reconfigure is guarded by hasattr,
a no-op where unavailable). Found and applied downstream in Perpetua-Tools;
contributing back.

Stacked on top of the episodic-mirror fix (atomic-01).
@diazMelgarejo

Copy link
Copy Markdown
Contributor Author

Related, independent PRs from the same review pass — each stands on its own, no dependency between them:

cc @codejunkie99 — happy to answer questions on any of these, or squash/reorder however's easiest to review.

@codejunkie99
codejunkie99 merged commit 9e0369e into codejunkie99:master Aug 6, 2026
1 check passed
codejunkie99 pushed a commit that referenced this pull request Aug 6, 2026
Patch release covering #60, #61, #62, #63: superseded-lesson filtering in
recall, the doubled skills/ path in upgrade's loop-skill copy, UTF-8 I/O in
learn.py, and a leaked file handle.

Collapses the stacked per-version README history (v0.9.0 through v0.18.0)
into a single pointer at CHANGELOG.md, and relaxes the onboarding docs test
from an exact version pin to the 0.19.x series so patch releases keep
asserting the loop docs and sandbox caveat without editing the assertion
each time.
cursor Bot pushed a commit to diazMelgarejo/Perpetua-Tools that referenced this pull request Aug 8, 2026


Bumps vendor/agentic-stack from 00eda65c (v0.9.0-era, unchanged since
2026-07-16) to 17f1bf65 (v0.19.0-9) -- 18 commits of genuine upstream
progress (bounded agentic loops, MiniMax provider, Autohand Code CLI
adapter) plus 3 PT-authored fixes not yet merged upstream:

- codejunkie99/agentic-stack#60 -- recall.py excludes superseded
  lessons (shared superseded_by_map helper with render_lessons.py)
- codejunkie99/agentic-stack#61 -- UTF-8 stdout/stderr + candidate
  writes in learn.py
- codejunkie99/agentic-stack#62 -- context manager in
  _lesson_already_appended

All 3 verified MERGEABLE with passing tests before pinning to them.
Also includes a 4th fix found and applied during this same pass:
harness_manager/upgrade.py's loop-skill copy doubled the "skills/"
path segment (.agent/skills/skills/loop-x/ instead of
.agent/skills/loop-x/), both in the CLI's own --dry-run report and the
real file copy -- silent because no existing test exercised a fresh
(non-pre-seeded) loop-skill destination. Fixed + regression test on
the fork; not yet opened as its own upstream PR.

Because none of the above have merged upstream yet, .gitmodules
temporarily points at the diazMelgarejo fork instead of
codejunkie99/agentic-stack -- documented inline with the exact revert
path (repoint the url, then
scripts/git/agentic-stack-submodule-sync.sh upgrade) once all 4 land.
Full provenance chain in
.agent/.agentic-stack-blend-state.json:last_blend.note.

Verified: a fresh clone of the fork successfully checks out 17f1bf6,
confirming `git submodule update --init` will resolve correctly for
anyone cloning PT with the current .gitmodules.
cursor Bot pushed a commit to diazMelgarejo/Perpetua-Tools that referenced this pull request Aug 8, 2026


Bumps vendor/agentic-stack from 00eda65c (v0.9.0-era, unchanged since
2026-07-16) to 17f1bf65 (v0.19.0-9) -- 18 commits of genuine upstream
progress (bounded agentic loops, MiniMax provider, Autohand Code CLI
adapter) plus 3 PT-authored fixes not yet merged upstream:

- codejunkie99/agentic-stack#60 -- recall.py excludes superseded
  lessons (shared superseded_by_map helper with render_lessons.py)
- codejunkie99/agentic-stack#61 -- UTF-8 stdout/stderr + candidate
  writes in learn.py
- codejunkie99/agentic-stack#62 -- context manager in
  _lesson_already_appended

All 3 verified MERGEABLE with passing tests before pinning to them.
Also includes a 4th fix found and applied during this same pass:
harness_manager/upgrade.py's loop-skill copy doubled the "skills/"
path segment (.agent/skills/skills/loop-x/ instead of
.agent/skills/loop-x/), both in the CLI's own --dry-run report and the
real file copy -- silent because no existing test exercised a fresh
(non-pre-seeded) loop-skill destination. Fixed + regression test on
the fork; not yet opened as its own upstream PR.

Because none of the above have merged upstream yet, .gitmodules
temporarily points at the diazMelgarejo fork instead of
codejunkie99/agentic-stack -- documented inline with the exact revert
path (repoint the url, then
scripts/git/agentic-stack-submodule-sync.sh upgrade) once all 4 land.
Full provenance chain in
.agent/.agentic-stack-blend-state.json:last_blend.note.

Verified: a fresh clone of the fork successfully checks out 17f1bf6,
confirming `git submodule update --init` will resolve correctly for
anyone cloning PT with the current .gitmodules.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants