real_api tests are not deselected by default; running the suite spawns real copilot subprocesses (can silently kill a live --web-bg workflow)

[jrob5756]

Summary

real_api-marked tests are not deselected by default. Despite their docstrings claiming they are "opt-in / deselected by default", the only thing that excludes them is the hand-written marker expression in the CI/release workflows. Any other invocation — a plain pytest, pytest -m "not performance", or make test — runs them, which spawns real copilot CLI subprocesses and makes real API calls.

This is more than a hygiene issue: it can hard-kill a live conductor run --web-bg workflow. When an agent (e.g. the sdd-implement coder) runs uv run pytest -m "not performance" from inside a running workflow, the real_api tests spin up competing copilot CLI subprocesses in the same process session as the workflow. That collides with the workflow's live Copilot session and the workflow process receives an unhandled SIGTERM — dying silently (no traceback, no faulthandler dump, PID file left behind, no workflow_failed event).

Reproduction

From a clean checkout (no real_api opt-in):

$ uv run pytest -q -m "not performance"
...
copilot._jsonrpc.JsonRpcError: JSON-RPC Error -32603: Request session.create failed
 with message: Model "claude-opus-4.7-1m-internal" is not available.
WARNING conductor.providers.copilot:copilot.py:644 Agent 'writer' attempt 3/3 failed: ...
...
FAILED tests/test_integration/test_copilot_large_write.py::test_large_create_tool_call_does_not_truncate
1 failed, 4030 passed, 27 skipped, 29 deselected, ... in 126.75s

test_copilot_large_write is marked @pytest.mark.real_api and its module docstring says it is "opt-in (real_api marker, deselected by default)" — yet it ran and made a real session.create call.

Evidence / root cause

• pyproject.toml [tool.pytest.ini_options] registers the real_api marker but sets no addopts to deselect it:

  [tool.pytest.ini_options]
  asyncio_mode = "auto"
  testpaths = ["tests"]
  markers = [
   "performance: ...",
   "real_api: marks tests that make real API calls to external services (deselect with '-m \"not real_api\"')",
   "install_scripts: ...",
  ]
  # no addopts -> real_api runs by default

• Only CI hand-excludes them:
  • .github/workflows/ci.yml: uv run pytest ... -m "not real_api and not performance"
  • .github/workflows/release.yml: uv run pytest -m "not real_api and not performance"
• Makefile test: target is uv run pytest -m "not install_scripts" — does not exclude real_api.
• @pytest.mark.real_api tests today include at least:
  • tests/test_integration/test_copilot_large_write.py
  • tests/test_integration/test_claude_real_api.py

Impact

1. Local make test / plain pytest is broken without real Copilot auth + the specific (internal) models available — a real_api test fails after 3 real retries.
2. Silent workflow kill: agents that run the test suite from inside a live --web-bg run get the workflow hard-killed with no error surfaced (observed twice on sdd-implement; diagnosis: unhandled SIGTERM, PID file survived, no faulthandler/workflow_failed).

Proposed fix

Make real_api truly opt-in via a conftest.py collection hook (robust: it can't be accidentally re-enabled by a stray -m "not performance" the way a bare addopts = -m ... can, since pytest keeps only the last -m):

# tests/conftest.py
def pytest_collection_modifyitems(config, items):
 marker_expr = config.getoption("-m") or ""
 if "real_api" in marker_expr:
 return # explicitly requested -> run them
 skip = pytest.mark.skip(reason="real_api test: opt in with -m real_api")
 for item in items:
 if "real_api" in item.keywords:
 item.add_marker(skip)

With this in place:

• make test, plain pytest, and the agent's pytest -m "not performance" all auto-skip real_api — no real Copilot subprocesses spawned, so no interference with a live workflow, and local runs go green.
• pytest -m real_api still runs them explicitly.
• CI's existing -m "not real_api and not performance" continues to work unchanged.

Optionally also fix the now-accurate docstrings and consider adding --cov-neutral guidance.

Environment

• Conductor v0.1.22
• Reproduced on Linux (WSL2), Python 3.12

[jrob5756]

Starting to take a look at this.