fix: make CI readiness and release evidence portable - #15
Merged
Conversation
…er detection
Two latent bugs surfaced when running the full suite together:
1. tests/test_architecture_isolation.py::_BlockCybernetic restored cascade
modules (agent_loop, tty_app) via sys.modules.update(self._blocked),
but the isolated tests re-import them first, so sys.modules ended up
holding a stale module object while subsequent `import minicode.agent_loop`
resolved a different one. This split broke monkeypatch.setattr in later
tests: the headless tests patched run_agent_turn on one object while
headless.py imported it from the other, so the patch silently no-op'd
and the real run_agent_turn ran against a mock `object()` adapter
('object' has no attribute 'next'). The two headless tests passed in
isolation but failed only in the full suite.
Fix: __exit__ drops cascade modules and re-imports them via
importlib.import_module so sys.modules and future imports agree on a
single canonical object.
2. detect_provider_name grew a runtime param threaded through every call
site, but its body called build_provider_config().provider, which
constructs a full ProviderConfig just to read a name and breaks test
doubles that stub only part of the provider-config surface. Switch to
the lightweight detect_provider() helper, which returns the Provider
enum directly while still honoring runtime-aware routing (custom
OpenAI-compatible hosts).
Verified: full suite 1167 passed, 2 skipped; model_switcher mypy clean.
Co-Authored-By: Claude <noreply@anthropic.com>…nfig model
Adds the custom-OpenAI-host probing feature that was sitting uncommitted in
the working tree, finishing it so the suite is green and CI's mypy baseline
(225) is not breached.
model_registry.py:
- Add list_openai_exposed_models / probe_openai_exposed_models /
model_is_exposed_via_openai_runtime. The probe hits the provider's
/v1/models endpoint, caches results keyed by (endpoint, api_key), and
writes the discovered list back into runtime["_openaiExposedModels"] so
later calls and config validation reuse it without re-fetching.
- detect_provider now routes a model to OPENAI when the runtime's custom
OpenAI-compatible host actually exposes it (e.g. a Claude model served
behind an OpenAI-compatible gateway).
- Use stdlib urllib instead of `requests` to avoid adding an untyped
third-party import that would inflate the mypy baseline (the rest of the
codebase uses urllib for the same reason).
config.py:
- validate_provider_runtime rejects a configured model that the custom
OpenAI-compatible provider does not expose, with actionable guidance
("Set `model` to one of the exposed models, or set `openaiExposedModels`
... if the exposed-model list is stale."). The guidance wording is what
makes the existing test_config_error_guidance assertion pass.
- describe_fallback_guidance surfaces the provider's currently-exposed
models when defaults already exist, so users know which models are
actually reachable on the custom host.
Tests updated to assert the exposed-models-aware behavior.
Verified: full suite 1165 passed, 2 skipped; mypy 225 (== baseline, no
regression); ruff clean on changed files.
Co-Authored-By: Claude <noreply@anthropic.com>…es; add hypothesis dep
Cross-platform compatibility fixes:
mcp.py (_prepare_spawn):
- On non-Windows, if 'python' is not in PATH but 'python3' is, auto-fallback
to python3. Many Linux distros only ship python3, which broke MCP server
startup with a cryptic 'Command not found: python' error.
run_command.py (_build_execution_command):
- Same python→python3 fallback for the run_command tool, which was causing
test_run_command_tool to fail on Linux.
tests/test_architecture_isolation.py:
- Add monkeypatch.setenv('ANTHROPIC_MODEL', ...) to test_core_config_without_cybernetic
so it does not depend on the developer's local settings.json.
pyproject.toml:
- Add hypothesis>=6.0.0 to dev dependencies (needed by test_property_based.py).
.gitattributes:
- Prevent future CRLF pollution: force LF line endings on all text files.
- Binary-mark package-lock.json files and nested git repos.
Test: 1165 passed, 2 skipped (live API tests, expected).
Co-Authored-By: Claude <noreply@anthropic.com>…l projection eval
Adds a comprehensive evaluation framework for Paper A (task-typed coding-agent
study):
minicode/:
- paper_a_task_completion_eval.py: full ablation harness — 5 conditions (none,
session-only, checkpoint-only, session+checkpoint, memory-backed) across 10
task families; produces structured JSON + markdown reports
- paper_a_external_projection_eval.py: out-of-sample evaluation using unseen
task prompts to test generalization
benchmarks/ (6 new eval scripts + 18 results files):
- paper_a_task_completion_eval.py / _results.{json,md}: main completion eval
- paper_a_task_completion_ablation_eval.py / _results.{json,md}: causal ladder ablation
- paper_a_continuity_eval.py / _results.{json,md}: session continuity eval
- paper_a_continuity_ablation_eval.py / _results.{json,md}: continuity ablation
- paper_a_external_projection_eval.py / _results.{json,md}: external projection
- paper_a_multisession_bridge_eval.py / _results.{json,md}: cross-session bridge
- paper_a_retrieval_probe_eval.py / _results.{json,md}: memory retrieval probe
tests/:
- test_paper_a_external_projection_eval.py: headless-based eval integration test
docs/:
- PAPER_PLAN.md: restructured to split Paper A (task-typed mainline) from
Paper B (cybernetic-memory reserve), with revised title direction, thesis
statement, and claim budget
Co-Authored-By: Claude <noreply@anthropic.com>mcp.py: add type: ignore for lambda calling close() returning None. tools/task.py: TypedDict for AGENT_TYPES; cast sub_messages to list[ChatMessage]. context_manager.py: fix cache_key type from int hash to str(hash(...)). smart_router.py: fix indexed assignment with local typed dict variable. Test: 1165 passed, 2 skipped. Co-Authored-By: Claude <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification