Skip to content

Fix transitive dependency handling in compile and orphan detection - #111

Merged
Daniel Meppiel (danielmeppiel) merged 10 commits into
mainfrom
copilot/fix-transitive-dependencies-issue
Feb 26, 2026
Merged

Fix transitive dependency handling in compile and orphan detection#111
Daniel Meppiel (danielmeppiel) merged 10 commits into
mainfrom
copilot/fix-transitive-dependencies-issue

Conversation

CopilotAI commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

🐛 Bug Fix

Problem

apm compile only scans direct dependencies from apm.yml, silently skipping transitive deps. Orphan detection (_check_orphaned_packages) and apm prune also only check apm.yml, incorrectly flagging transitive deps as orphaned.

Example: package A → B → C. Only A is in apm.yml. After apm install, B and C are installed and tracked in apm.lock with depth: 2+, but apm compile ignores their primitives and reports them as orphans.

Solution

Read apm.lock (which already tracks transitive deps with depth info) in addition to apm.yml in the three affected code paths:

  • get_dependency_declaration_order() (discovery.py): Appends transitive deps from lockfile after direct deps, preserving priority ordering
  • _check_orphaned_packages() (cli.py): Includes lockfile deps in expected_installed set
  • prune() command (cli.py): Same lockfile inclusion for expected deps
  • get_lockfile_installed_paths() (lockfile.py): New helper that converts LockedDependency entries to their filesystem install paths via DependencyReference.get_install_path(), handling regular, virtual, and ADO package layouts

Testing

  • Bug reproduction confirmed
  • Fix verified locally
  • Regression tests added (if applicable)
  • All existing tests pass

17 new unit tests across two files:

  • tests/unit/test_transitive_deps.py — lockfile path resolution, multi-level transitive discovery, orphan detection with/without lockfile
  • tests/test_enhanced_discovery.py — 4 additional tests for transitive primitive scanning

817 total unit tests pass, CodeQL clean.

Checklist

  • LABEL: Apply bug or fix label to this PR
  • Root cause identified and addressed
  • Edge cases considered
Original prompt

This section details on the original issue you should resolve

<issue_title>[BUG] transitive packages</issue_title>
<issue_description>Describe the bug
I don't have the same result if I use transitive dependencies or direct dependencies of the same packages.

dependencies

  • package team-cot-agent-instructions
name: team-cot-agent-instructionsversion: 0.0.1description: the instructions of the Centralized Operation Team of the IME divisiondependencies:
apm:
- git.autodesk.com/rieraj/division-ime-agent-instructions
  • package division-ime-agent-instructions
name: division-ime-agent-instructionsversion: 0.0.1description: This document is the main page of the instructions of the Interactive Media Entertainment division (IME)dependencies:
apm:
- git.autodesk.com/rieraj/autodesk-agent-instructions
  • package autodesk-agent-instructions
name: autodesk-agent-instructionsversion: 0.0.1description: This document is the main page of the instructions of the Autodesk company.

apm.yml files

  • direct dependencies
name: drectversion: 1.0.0description: directauthor: Jordi Rieradependencies:
apm:
- git.autodesk.com/rieraj/team-cot-agent-instructions
- git.autodesk.com/rieraj/division-ime-agent-instructions
- git.autodesk.com/rieraj/autodesk-agent-instructionsmcp: []scripts: {}
  • transitive dependencies
name: transitivesversion: 1.0.0description: APM project for transitivesauthor: Jordi Rieradependencies:
apm:
- git.autodesk.com/rieraj/team-cot-agent-instructionsmcp: []scripts: {}

To Reproduce
Steps to reproduce the behaviour:

  1. Run command 'apm install' in both cases
  2. With parameters 'None'
  3. See error

Expected behaviour

In the case of direct dependencies

❯ apm install
Installing dependencies from apm.yml...
Installing APM dependencies (3)...
└─ Resolved transitive: rieraj/team-cot-agent-instructions
└─ Resolved transitive: rieraj/division-ime-agent-instructions
└─ Resolved transitive: rieraj/autodesk-agent-instructions
Created .github/ as standard skills root (.github/skills/) and to enable
VSCode/Copilot integration
✓ rieraj/autodesk-agent-instructions
✓ rieraj/division-ime-agent-instructions
✓ rieraj/team-cot-agent-instructions
└─ 1 agents integrated → .github/agents/
Generated apm.lock with 3 dependencies
Installed 3 APM dependencies
No MCP dependencies found in apm.yml
╭─────────────────────────── ✨ Installation complete ───────────────────────────╮
│ │
│ Next steps: │
│ apm compile # Generate AGENTS.md guardrails │
│ apm list # Show all prompts │
│ │
╰────────────────────────────────────────────────────────────────────────────────╯
❯ apm compile
⚙️ Starting context compilation...
Compiling for AGENTS.md (VSCode/Copilot) - detected .github/ folder
Analyzing project structure...
├─ 4 directories scanned (max depth: 3)
├─ 9 files analyzed across 3 file types (lock, md, yml)
└─ 3 instruction patterns detected
Optimizing placements...
Pattern Source Co… Placement Metrics
────────────────────────────────────────────────────────────────────────────────
** IME-ruler.in… 4/4 ./AGENTS.md rel: 100%
** design-stand… 4/4 ./AGENTS.md rel: 100%
** autodesk-sta… 4/4 ./AGENTS.md rel: 100%
Generated 1 AGENTS.md file
┌─ Context efficiency: 100.0%
└─ Generation time: 4ms
Placement Distribution
└─ . 3 instructions from 2 sources
✅ Compilation completed successfully!

In the case of transitive dependencies

❯ apm install
Installing dependencies from apm.yml...
Installing APM dependencies (1)...
└─ Resolved transitive: rieraj/team-cot-agent-instructions
└─ Resolved transitive: rieraj/division-ime-agent-instructions
└─ Resolved transitive: rieraj/autodesk-agent-instructions
Created .github/ as standard skills root (.github/skills/) and to enable
VSCode/Copilot integration
✓ rieraj/team-cot-agent-instructions
└─ 1 agents integrated → .github/agents/
✓ rieraj/division-ime-agent-instructions
✓ rieraj/autodesk-agent-instructions
Added apm_modules/ to .gitignore
Generated apm.lock with 3 dependencies
Updated .gitignore for integrated agents (*-apm.agent.md, *-apm.chatmode.md)
Installed 3 APM dependencies
No MCP dependencies found in apm.yml
╭─────────────────────────── ✨ Installation complete ───────────────────────────╮
│ │
│ Next steps: ...
</details>
<!-- START COPILOT CODING AGENT SUFFIX -->
- Fixes microsoft/apm#110
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

CopilotAIand others added 2 commits February 25, 2026 23:21
Read from apm.lock (which tracks transitive deps) in addition to apm.yml:
- get_dependency_declaration_order() now includes transitive deps for compile
- _check_orphaned_packages() no longer flags transitive deps as orphaned
- prune command accounts for transitive deps in expected_installed set
- Added get_lockfile_installed_paths() helper to lockfile module
- Added 17 unit tests covering all transitive dependency scenarios
Co-authored-by: danielmeppiel <51440732+danielmeppiel@users.noreply.github.com>
- Wrap get_lockfile_installed_paths() in try/except to handle all errors
- Use ImportError instead of broad Exception in callers
- Update docs for transitive dep discovery
Co-authored-by: danielmeppiel <51440732+danielmeppiel@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix issue with transitive package dependenciesFix transitive dependency handling in compile and orphan detectionFeb 25, 2026
@danielmeppielDaniel Meppiel (danielmeppiel) added the bug Deprecated: use type/bug. Kept for issue history; will be removed in milestone 0.10.0. label Feb 25, 2026
@danielmeppielDaniel Meppiel (danielmeppiel) linked an issue Feb 25, 2026 that may be closed by this pull request
@danielmeppiel
Daniel Meppiel (danielmeppiel) marked this pull request as ready for review February 25, 2026 23:50
CopilotAI review requested due to automatic review settings February 25, 2026 23:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes apm compile, orphan detection, and apm prune so they account for transitive dependencies by incorporating apm.lock (in addition to apm.yml) when determining expected installed dependency paths.

Changes:

  • Extend dependency discovery order to append transitive deps from apm.lock after direct deps from apm.yml.
  • Update orphan detection and prune logic to treat lockfile-listed deps as expected (avoiding false orphan reports).
  • Add a lockfile helper for resolving installed paths and introduce targeted unit/integration tests + docs update.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
FileDescription
tests/unit/test_transitive_deps.pyAdds unit tests covering lockfile path resolution, transitive ordering, and orphan detection behavior.
tests/test_enhanced_discovery.pyExpands discovery tests to validate transitive dependency inclusion and primitive scanning.
src/apm_cli/primitives/discovery.pyAppends transitive dependency install paths from apm.lock to discovery order.
src/apm_cli/deps/lockfile.pyAdds get_lockfile_installed_paths() helper to convert lock entries into install paths.
src/apm_cli/deps/init.pyExposes get_lockfile_installed_paths as part of the deps package API.
src/apm_cli/cli.pyUpdates orphan detection and prune to include lockfile dependencies as expected installs.
docs/enhanced-primitive-discovery.mdDocuments updated behavior: direct deps from apm.yml, transitives appended from apm.lock.

Comment threadsrc/apm_cli/deps/lockfile.py
Comment threadsrc/apm_cli/deps/lockfile.py Outdated
Comment threadsrc/apm_cli/primitives/discovery.py Outdated
Comment threadtests/unit/test_transitive_deps.py Outdated
Comment threadsrc/apm_cli/deps/lockfile.py
Comment threadsrc/apm_cli/deps/lockfile.py
Comment threadsrc/apm_cli/primitives/discovery.py Outdated
Comment threadsrc/apm_cli/cli.py Outdated
- Move get_lockfile_installed_paths logic into LockFile.get_installed_paths()
method (foutoucour review)
- Use .as_posix() for Windows-safe path normalization (copilot-reviewer)
- Narrow except Exception to specific error types (copilot-reviewer)
- Move inline imports to top-level in discovery.py and cli.py (foutoucour)
- Remove unused imports in test file (tempfile, pytest)
@danielmeppiel

Daniel Meppiel (danielmeppiel) commented Feb 26, 2026

Copy link
Copy Markdown
Collaborator

Summary of additional fixes since initial PR

Beyond the original transitive dependency fix, this PR now includes several follow-up fixes addressing review feedback and CI issues:

1. deps list console width fix (48d87c0)

CI was failing on test_deps_list_shows_correct_path and test_mixed_deps_list in ADO E2E tests. Root cause: PR #107 added 4 new columns (Prompts, Instructions, Agents, Skills) to the deps list table, expanding it from 5 to 7 columns. At CI's default 80-col terminal, Rich truncated the Package and Source columns, breaking assertions.
Fix: Set Console(width=max(120, term_width)) in list_packages() to guarantee minimum 120-col rendering.

2. LockFile classmethod refactor (bd8794c)

Per review feedback, refactored the standalone get_lockfile_installed_paths() function into a LockFile.installed_paths_for_project() classmethod — better cohesion since it operates on lockfile data. Updated all 4 callers (cli.py ×2, discovery.py ×1, test_transitive_deps.py ×7 call sites). Old function kept as a deprecated one-line alias for backward compat.

3. Integration test assertion fix (640ebbb)

test_skill_detection_in_output was asserting for "Claude Skill" or "SKILL.md detected" in CLI output, but the current output format is "Skill integrated → .github/skills/". Updated assertion to match.

Test verification

  • 812 unit tests — all pass ✅
  • 9 ADO E2E tests — all pass ✅
  • 30 integration tests (the 4 test files that were previously failing) — all pass ✅

The "Integration Tests (PR) — Integration tests failed" will keep showing until the above CI fixes (1 and 3) are merged with this PR.

@danielmeppiel
Daniel Meppiel (danielmeppiel) merged commit 5aea357 into mainFeb 26, 2026
5 of 6 checks passed
@danielmeppiel
Daniel Meppiel (danielmeppiel) deleted the copilot/fix-transitive-dependencies-issue branch February 27, 2026 09:41
Sergio Sisternes (sergio-sisternes-epam) pushed a commit that referenced this pull request May 19, 2026
…ncies-issue
Fix transitive dependency handling in compile and orphan detection
Daniel Meppiel (danielmeppiel) added a commit that referenced this pull request May 27, 2026
Address 9 Copilot reviewer findings (A1-A9) + 5 Deploy Docs link
failures (B1-B5) on the OpenAPM v0.1 spec:
Schemas
- policy: deny/require accept null (parity with allow) (A1)
- lockfile: comment -> $comment annotation (A2)
- manifest: registries object-form allows x-* extensions (A9)
Spec prose
- Retarget broken anchors: #2-conventions (A3/B4),
#111-conformance-classes-normative (B1/B2), 10.6 -> 10.7 (B3),
#422-type-advisory (B5)
- Appendix E.1 type set reconciled to instructions/skill/hybrid/prompts (A5)
- Field name targets -> target in two §8.4 examples (A6)
- §7.3.1 wildcard row: drop 'latest' (semver-only) (A7)
- Appendix C req-mf-004 MUST -> SHOULD matches body prose (A4)
CHANGELOG
- Recover dropped v0.16-cycle Unreleased entries lost in branch
rebase (A8): git-semver ref:, apm deps why, --update fix,
=1.2.3 pinned-classification
Local Starlight build: zero links-validator errors, 111 pages built.
Skill: apm-spec-guardian
- New advisory panel skill institutionalizing the 4-persona spec
review process used for this saga
- Activates on PRs touching docs/src/content/docs/specs/openapm-*.md,
schemas/*.schema.json, or tests/fixtures/spec-conformance/**
- 5 personas materialized as .apm/agents/spec-*.agent.md:
swagger-editor, oci-editor, pkgmgr-editor, tag-architect,
editor-synthesizer
- Single-writer interlock, advisory-only (no merge gate), shocked-meter
ship signal, 11-check linter checklist mirrored from this saga
- Designed via genesis discipline (design pack in session-state)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Daniel Meppiel (danielmeppiel) added a commit that referenced this pull request May 28, 2026
* spec: introduce OpenAPM v0.1 (closes#1502)
OpenAPM v0.1 is a single normative specification document at
docs/src/content/docs/specs/openapm-v0.1.md (2678 lines, 87 normative
statements). It pins the contract surface that current and future
integrators consume (apm.yml manifest, apm.lock.yaml lockfile,
apm-policy.yml governance), isolating ecosystem tooling from
URL-level churn in this implementation.
Ships:
- The specification document (87 req-XXX statements, four
conformance classes: Producer / Consumer / Registry / Governance)
- Inline JSON Schemas (Draft 2020-12) for manifest, lockfile, policy
at docs/src/content/docs/specs/schemas/
- Conformance fixture seed tree at tests/fixtures/spec-conformance/
(10 files including semver-dialect resolution oracle, valid/invalid
manifests, lockfiles across v1 and v2, and policy extends cases)
- CHANGELOG entry
Vendor-neutral (no specific standards body mentioned), MIT-licensed
editor's draft under semver-zero. Self-contained: a third-party
integrator can build a conformant implementation without reading
any other APM doc.
Review process: passed two rounds of 4-panel adversarial review
(OpenAPI/Swagger, OCI Distribution, npm/cargo registry contract,
W3C TAG architect personas). Round-2 verdict: unanimous
ship_with_followups at shocked_meter 8/10, zero new blocking
findings. v1.1 fold pass closed 18 editorial / defensive-MUST /
internal-consistency items surfaced in round 2.
Reserved for v0.2 (called out in the artifact): workspaces (s4.8),
version withdrawal / yank (s7.9), publisher attestations and
signatures (s10.12), Registry HTTP API (Appendix B), machine-readable
conformance manifest (s12.6), frozen-default flip (s5.5),
canonical-tree edges for symlinks/submodules/LFS (s5.6.4),
reproducible-build determinism (s1.1 non-goal), update --aggressive
(s7.7), marketplace authoring output (s4.7).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(spec): address PR #1517 review + introduce apm-spec-guardian skill
Address 9 Copilot reviewer findings (A1-A9) + 5 Deploy Docs link
failures (B1-B5) on the OpenAPM v0.1 spec:
Schemas
- policy: deny/require accept null (parity with allow) (A1)
- lockfile: comment -> $comment annotation (A2)
- manifest: registries object-form allows x-* extensions (A9)
Spec prose
- Retarget broken anchors: #2-conventions (A3/B4),
#111-conformance-classes-normative (B1/B2), 10.6 -> 10.7 (B3),
#422-type-advisory (B5)
- Appendix E.1 type set reconciled to instructions/skill/hybrid/prompts (A5)
- Field name targets -> target in two §8.4 examples (A6)
- §7.3.1 wildcard row: drop 'latest' (semver-only) (A7)
- Appendix C req-mf-004 MUST -> SHOULD matches body prose (A4)
CHANGELOG
- Recover dropped v0.16-cycle Unreleased entries lost in branch
rebase (A8): git-semver ref:, apm deps why, --update fix,
=1.2.3 pinned-classification
Local Starlight build: zero links-validator errors, 111 pages built.
Skill: apm-spec-guardian
- New advisory panel skill institutionalizing the 4-persona spec
review process used for this saga
- Activates on PRs touching docs/src/content/docs/specs/openapm-*.md,
schemas/*.schema.json, or tests/fixtures/spec-conformance/**
- 5 personas materialized as .apm/agents/spec-*.agent.md:
swagger-editor, oci-editor, pkgmgr-editor, tag-architect,
editor-synthesizer
- Single-writer interlock, advisory-only (no merge gate), shocked-meter
ship signal, 11-check linter checklist mirrored from this saga
- Designed via genesis discipline (design pack in session-state)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fold spec drift-detection scaffolding (apm-spec-guardian dictated)
PR #1517 introduced the OpenAPM v0.1 specification but APM is its sole
implementation. Without an in-tree drift detector the spec rots silently
across three modes: (A) silent regression -- a code change breaks a
requirement no test catches; (B) silent extension -- APM gains a behaviour
that should be normative but never enters the spec; (C) stale spec -- the
prose disagrees with the intended behaviour and nobody notices.
This fold is the dictate from the apm-spec-guardian skill panel
(swagger / OCI / pkgmgr / TAG editors, synthesised) red-teaming a v0.1.1
drift-detection design memo. Synthesizer recommendation:
ship_decision=next_brief, fold F1..F11.
Folded:
* F1 machine-readable requirements manifest at
docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
(informative in v0.1.1; reserved for normative promotion in v0.2 per
spec sec.12.6).
* F2 JSON Schema 2020-12 contract for the manifest at
docs/src/content/docs/specs/schemas/requirements-v0.1.schema.json.
* F3 tests/spec_conformance/orphan_check.py: 4-way bind between the
canonical spec body anchors, the manifest, Appendix C, and the
pytest @pytest.mark.req coverage. CI fails on any divergence.
* F4 tests/spec_conformance/conftest.py: registers the `req` marker
and validates it against the manifest at collection time. Records
a static-analysis status (active / skipped / xfail) at
build/conformance-coverage.json.
* F5 tests/spec_conformance/test_{manifest,lockfile,policy,resolution,
round_trip,gen_statement}_reqs.py: 87 markers covering every
requirement. 27 active assertions against the seed fixtures and
shipped schemas; 69 honest waivers ("waiver: <reason>") for
behaviours that need v0.1.2 fixture expansion or producer harness
surface. The conformance statement surfaces the gap legibly.
* F6 tests/spec_conformance/gen_statement.py: writes a deterministic
CONFORMANCE.json + CONFORMANCE.md at repo root (canonical, sorted,
ASCII-only, LF line endings, generator-stamped).
* F7 tests/spec_conformance/test_gen_statement.py: byte-equal
regeneration, ASCII-only assertion, version+generator stamps, and
the literal "NO automated CI detector" honesty phrase.
* F8 tests/spec_conformance/test_round_trip.py: stage-2 fixed-point
parametrised over five round-trip fixtures (manifest, lockfile,
policy). Honours `round_trip_exempt_fields` from the manifest.
* F9 .github/workflows/spec-conformance.yml: paths-triggered gate that
runs orphan_check, the conformance pytest suite, regenerates
CONFORMANCE.{json,md}, and gates on `git diff --exit-code` so
contributors must commit any regeneration.
* F10 spec sec.12.3 rewritten to identify the HTML anchors as canonical
source (manifest and Appendix C are derived projections); sec.12.6
reservation preserved with informative v0.1.1 companion paragraph
appended; Appendix D row updated.
* F11 CONFORMANCE.md preamble carrying the literal honesty contract;
CONTRIBUTING.md "Adding or changing a normative requirement"
three-step ritual; .github/pull_request_template.md spec-
conformance checklist enforcing the ritual at PR time.
Anti-patterns explicitly rejected per the synthesizer (do NOT reintroduce):
one pytest file per req; an `apm conformance run` user command; 100%
line coverage as the conformance bar; Appendix-C-table-regex as load-
bearing CI source of truth.
Maintainer mandate: ship with the spec, NOT as a follow-up PR.
Verified locally:
$ uv run --extra dev python -m tests.spec_conformance.orphan_check
[+] orphan_check OK: 87 requirements aligned across anchors / manifest
/ Appendix C / pytest markers
$ uv run --extra dev pytest tests/spec_conformance -p no:randomly
27 passed, 69 skipped
$ uv run --extra dev ruff check src/ tests/
All checks passed!
$ uv run --extra dev ruff format --check src/ tests/
1134 files already formatted
$ uv run --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/
Your code has been rated at 10.00/10
$ bash scripts/lint-auth-signals.sh
[+] auth-signal lint clean
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* spec: convert 68 conformance waivers to active assertions (apm-spec-guardian directive)
Closes the spec-guardian's full-completeness directive on PR #1517:
ship the v0.1 spec with active conformance coverage on every
testable requirement (in-PR, not in a future version).
## Coverage shift
Before: 27 active / 69 skipped / 87 total (sole-implementor risk).
After: 86 active / 1 skipped / 87 total.
Per class:
Producer 12 / 12 active
Consumer 61 / 62 active (1 honest waiver: req-mf-016)
Registry 1 / 1 active (was: waived entirely)
Governance 12 / 12 active
The sole remaining skip is req-mf-016 (apm-source absolute-path
rejection), which requires the path-policy loader to be invokable
from the harness; the schema currently models `path` as free-form
string and the waiver text documents the tightening as the next
schema-side follow-up.
## Conversion strategies used
1. **Schema introspection**: read the manifest/lockfile/policy JSON
Schemas, assert the structural invariants they encode (depEntry
oneOf, hashEnvelope pattern, registries https-only, hash_algorithm
strong-set, pattern properties for x-* extensions, etc.).
2. **Spec-text grep** (`assert_spec_contains`): assert that the
verbatim normative phrasing remains in the spec body. Catches
silent-deletion drift: if the authoring panel rewords or drops a
clause without updating the assertion, the suite fails at PR time.
3. **Negative fixtures**: 4 new manifest fixtures
(invalid-registry-scheme, invalid-registries-typo,
invalid-source-kind, invalid-yaml-anchor-alias) drive jsonschema
validation to failure on the boundary the spec forbids.
4. **Integrity oracle (req-rg-001, req-lk-012..017)**: a
deterministic 176-byte tar.gz archive
(`integrity/security-baseline-2.3.1.tar.gz`) plus four paired
lockfile fixtures (canonical, hash-mismatch, deployed-file-mismatch,
bare-hex-reader). The registry trust-anchor test reads the archive
bytes, recomputes SHA-256, and asserts equality with the lockfile's
advertised digest. Tampering with either side breaks the bind. The
mismatch fixtures prove the fail-closed contract is observable.
5. **apm_cli loader integration**: req-pl-005/007/008 drive
`apm_cli.policy.parser.load_policy` against the valid-extends
fixture and assert the ApmPolicy model carries the expected
allow/deny tuples.
6. **Oracle parametrisation**: req-rs-008 (caret), req-rs-009
(tilde), req-rs-010 (exact), req-rs-014 (prerelease/build) all
walk `resolution/semver-dialect.json` as parametrised cases.
7. **Literal regex validation**: req-pr-004 (git-semver tag form)
compiles the literal regex from sec.8.5 and validates it against
10 positive + negative tag examples.
## New files
tests/fixtures/spec-conformance/integrity/
security-baseline-2.3.1.tar.gz (Registry oracle)
security-baseline-2.3.1.frozen.yaml (req-rg-001, req-lk-006/010/012)
hash-mismatch.frozen.yaml (req-lk-013)
deployed-file-mismatch.frozen.yaml (req-lk-017)
bare-hex-reader.frozen.yaml (req-lk-016)
tests/fixtures/spec-conformance/manifest/
invalid-registry-scheme.yml (req-mf-014, req-sc-007)
invalid-registries-typo.yml (req-mf-015)
invalid-source-kind.yml (req-mf-012)
invalid-yaml-anchor-alias.yml (req-mf-020)
tests/spec_conformance/test_registry_reqs.py (req-rg-001 active)
## Helpers
_helpers.py: + assert_spec_contains(*needles), spec_text() cache,
sha256_hex() for integrity oracles.
## Doc deltas
gen_statement.py: rewrite the 'Conformance classes' preamble to
declare active coverage on all four classes (Producer, Consumer,
Registry, Governance); the registry class is exercised through
the trust-anchor SHA-256 invariant test.
CONTRIBUTING.md: replace 'bump 0.1.1 -> 0.1.2' phrasing in the
Mode C bullet with version-agnostic 'bump the spec patch
revision' (no implicit version-bumping commitment).
## Verification
uv run --extra dev pytest tests/spec_conformance -p no:randomly
-> 109 passed, 2 skipped in 4.28s (skips = req-mf-016 waiver +
req-lk-018 publisher SHOULD with a recorded waiver alongside
the schema-affordance assertion)
uv run --extra dev python -m tests.spec_conformance.orphan_check
-> 87 requirements aligned across the 4-way bind
uv run --extra dev python -m tests.spec_conformance.gen_statement
-> regenerates CONFORMANCE.{md,json} (86 active / 1 skipped)
uv run --extra dev ruff check src/ tests/ : OK
uv run --extra dev ruff format --check src/ tests/ : OK
uv run --extra dev pylint --enable=R0801 src/apm_cli/ : 10/10
bash scripts/lint-auth-signals.sh : clean
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* spec(v0.1.2): Mode B silent-extension detector + round-3 editorial fold
Closes the sole-implementer rot risk the maintainer named on PR #1517
review: the existing 4-way orphan_check enforces equality among already-
DECLARED artifacts (anchors / manifest / Appendix C / pytest markers)
but cannot detect new APM behaviour under critical paths with zero
markers. This commit adds a complementary CI gate plus folds 7
spec-guardian editorial findings.
Mode B detector
---------------
* tests/spec_conformance/mode_b_detector.sh: shell gate. Counts
substantive added lines under the critical-path allowlist
(primitives/, deps/, policy/, registry/, runtime/, install/,
integration/), short-circuits when the PR is spec-concurrent (touches
the spec body, requirements manifest, or adds a new @pytest.mark.req
marker), and fires with an actionable diagnostic when the threshold
is crossed without a citation.
* tests/spec_conformance/critical_paths.txt: editable allowlist; edits
are themselves critical-path edits per CONTRIBUTING.md.
* .github/workflows/spec-conformance.yml: wires the detector as a step
after orphan_check.
* Waiver mechanism: 'apm-spec-waiver: <reason, >= 16 chars>' in the PR
body OR a commit message between merge-base and HEAD. Echoed to CI.
* CONTRIBUTING.md Mode B section rewritten honestly (no longer claims
orphan_check catches silent extension on its own).
Round-3 editorial fold (no new req IDs; statement count stays at 87)
--------------------------------------------------------------------
* Section 11.3.2 Consumer enumeration: appended req-rs-014 and
req-cf-002 (closes drift vs Appendix C, sw-rec-r1-1).
* req-lk-005: writers MUST canonicalise dependencies list in ascending
(repo_url, virtual_path) order; frozen-install diffs now stable
across implementations (pkg-rec-r1-1).
* req-sc-003: consumers MUST drop the originating Authorization header
across cross-host-class redirects, closing the mirror-redirect
token-leak surface (oci-rec-r1-4).
* req-rg-001: extended with publish-side idempotency clause -- Registry
MUST either reject republish or accept ONLY byte-identical bytes
(oci-rec-r1-5).
* Section 6.2 + 6.3.1 defaults pinned: fetch_failure defaults to warn;
dependencies.require_resolution defaults to project-wins. Mirrored
as advisory "default" annotations in policy-v0.1.schema.json
(pkg-rec-r1-5).
* manifest-v0.1.schema.json: conflict_resolution enum aligned to prose
(intersect -> intersection-pick); nest dropped from v0.1 enum (still
reserved for v0.2 per req-rs-013, now noted via schema $comment)
(pkg-rec-r1-2).
* lockfile-v0.1.schema.json: stripped internal review-token leak
'(see oci-r2-nit-2)' from $comment (oci-nit-r1-1).
* Appendix D: 0.1.2 revision row added documenting all of the above.
Test coverage
-------------
Mode-C-style normative-phrase pins added for the 3 strengthened reqs
(req-lk-005, req-sc-003, req-rg-001) so the new clauses cannot
silently disappear from the spec body. All 109 conformance tests pass;
orphan_check aligned at 87 requirements.
Verification: ruff/pylint R0801/auth-signals all green; orphan_check
clean; pytest tests/spec_conformance 109 passed, 2 skipped.
Refs: PR #1517
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(spec-conformance): self-test the Mode B detector + pin round-3 schema changes
Closes three gaps surfaced by the gate-completeness audit on top of
the round-3 spec-drift fold:
A. Mode B detector self-tests (NEW test_mode_b_detector.py, 9 cases)
The detector script was itself honor-system -- a future edit could
silently break the gate logic and CI would not notice. The new
suite pins the script's structural contract (exists, executable,
bash -n clean), the critical-path allowlist shape (every entry
resolves to a real dir; primitives/deps/policy/registry are all
covered), the CI workflow wiring (orphan_check -> mode_b_detector
-> conformance suite ordering), AND the gate behaviour itself by
running the script via subprocess inside a synthetic git repo:
- spec-concurrent edit short-circuits (exit 0)
- out-of-scope only passes (exit 0)
- substantive critical-path add with no spec edit fires (exit 1)
- commit-trailer waiver accepted
- short waiver (<16 chars) rejected
B. Round-3 schema/default pins
test_resolution_reqs.py req-rs-013 now asserts the manifest schema
enum is exactly [intersection-pick] (not just that the spec prose
names it); a reverter to [intersect, nest] now trips a test.
test_policy_reqs.py req-pl-010 now asserts the fetch_failure and
require_resolution defaults are pinned in the schema AND in spec
prose.
C. Workflow trigger expansion
Added CONTRIBUTING.md and .github/workflows/spec-conformance.yml
to the spec-conformance paths filter so edits to the Mode B
narrative or the gate itself are re-verified by the gate.
Verification: ruff/format clean, pylint R0801 10.00/10, auth-signals
clean, orphan_check OK 87 reqs, pytest 118 passed / 2 skipped (the
two documented absolute-path / lockfile-sort waivers).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(spec): fold OpenAPM v0.1 into the published Starlight site
The spec page builds at /apm/specs/openapm-v01/ but was undiscoverable
and its declared schema $id URLs were unreachable. Designed with a
Starlight + spec-publishing expert (OpenAPI / JSON Schema / OCI
precedent) and the APM doc-writer; folded end-to-end so the spec ships
with PR #1517 instead of leaking out half-published.
Five changes:
1. Move normative artifacts to docs/public/specs/{schemas,manifests}/
so the declared $id URLs (https://microsoft.github.io/apm/specs/
schemas/<name>.schema.json) resolve to byte-identical files on the
built site. Schemas and YAML lived under the Starlight content
collection, where Astro does not serve them as static assets --
the $ids were a promise the site could not keep. Now they ship at
the canonical URL, exactly as JSON Schema, OpenAPI, CycloneDX, and
SPDX serve theirs.
Tooling paths updated in lockstep: _manifest.py, _helpers.py,
bootstrap_requirements.py, mode_b_detector.sh, test_mode_b_detector.
Also fixed a real bug surfaced by the new gate: requirements-v0.1
.schema.json had $id pointing at github.com/microsoft/apm/spec/...
instead of microsoft.github.io/apm/specs/... (would have landed any
pinning toolchain off-site).
2. New top-level 'Specification' sidebar group ABOVE 'CLI reference'
in docs/astro.config.mjs. Two entries: the spec page and a new
thin Conformance wrapper page that points to CONFORMANCE.md and
.json at the repo root. The spec is the centerpiece normative
artifact; placement signals that.
3. Stable shortlink redirects /spec, /spec/latest, /spec/v0.1 ->
/apm/specs/openapm-v01/ for external citation. Mirrors OpenAPI
and AsyncAPI URL discipline: versioned URLs are immortal and
pin-targetable; /latest is for human prose only. Documented on
the spec page in a new 'Citing this specification' subsection
that explicitly says toolchains MUST NOT pin to /latest.
4. Cross-link banners on reference/{manifest-schema,lockfile-spec,
policy-schema}.md point readers at the normative v0.1 spec
sections (4 / 5 / 6) and at the canonical schema $id URLs.
Resolves the silent-drift gap where a reader landing on the v0.2
working-draft reference pages had no signal that a ratified v0.1
normative spec existed.
5. New CI gate scripts/check_schema_ids.py wired into
.github/workflows/docs.yml after the docs build step. For every
schema under docs/public/specs/schemas/, it asserts: (a) the
declared $id is on-site under /apm/, (b) the path exists in
docs/dist/, (c) the bytes are sha256-identical to the source.
Catches schema-moved-without-$id-update, $id typos, and
Starlight accidentally consuming a public/ asset. Pure stdlib so
it runs on the existing Node-only docs job without an extra
Python setup.
Plus a 60-word landing blurb at the top of the spec orienting
first-time readers ('the consumer/producer/enterprise ramps tell you
how to USE APM; this page is the contract that defines what APM IS').
Verification:
- npm run build: 111 pages, all internal links valid
- check_schema_ids.py: 4/4 schemas resolve sha256-identical
- orphan_check: 87 reqs aligned
- pytest tests/spec_conformance: 118 passed, 2 skipped
- ruff/format/pylint R0801/auth-signals: all green
- dist/spec/{,v0.1,latest}/index.html: meta-refresh redirects shipped
Honest gaps left for follow-up (out of scope here):
- multi-version /latest routing is manual until v0.2 lands
- section heading IDs not yet stabilized for deep-link survival
- normative-vs-informative visual treatment (MUST/SHOULD styling)
- schemastore.org registration
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: danielmeppiel <danielmeppiel@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugDeprecated: use type/bug. Kept for issue history; will be removed in milestone 0.10.0.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] transitive packages

5 participants

@danielmeppiel@foutoucour@SebastienDegodez