Skip to content

feat(registry): tag-based versioning with # ref syntax and CDN-bypass - #151

Merged
Jason Robert (jrob5756) merged 2 commits into
mainfrom
feat/registry-tag-versioning
May 6, 2026
Merged

feat(registry): tag-based versioning with # ref syntax and CDN-bypass#151
Jason Robert (jrob5756) merged 2 commits into
mainfrom
feat/registry-tag-versioning

Conversation

@jrob5756

Copy link
Copy Markdown
Collaborator

Summary

Reworks the workflow registry to behave more like a plugin system — versions are auto-discovered from git tags instead of being explicitly listed, refs can pin to any tag/branch/SHA, and stale CDN content is no longer a problem.

Closes the two pain points raised in discussion:

  1. `conductor registry update` doesn't reflect changes immediately — root cause was Fastly's CDN cache on `raw.githubusercontent.com`. Index fetches now resolve the ref to a commit SHA via the GitHub API and fetch from `…//index.yaml`. Unique URL per commit → CDN can't serve stale data. No `--force` flag needed.

  2. Versioning was awkward — required maintainers to enumerate every release in `index.yaml`. Now versions are auto-discovered from the registry repo's git tags, semver-sorted.

New ref grammar

```
[@][#]
```

ExampleResolves to
`qa-bot`default registry, latest tag (or default branch HEAD if no tags)
`qa-bot#v1.2.3`default registry, ref `v1.2.3`
`qa-bot#main`default registry, branch `main`
`qa-bot@team`registry `team`, latest
`qa-bot@team#v1.2.3`registry `team`, tag `v1.2.3`
`qa-bot@#abc123`default registry, commit `abc123`

Hard errors: empty ref after `#`, multiple `#` or `@`, empty workflow name, path registry with a ref.

Behavior changes

  • `versions: [...]` field dropped from `index.yaml` schema. Existing indexes with this field still parse (Pydantic ignores it) — no migration required.
  • `latest` = newest semver-sorted tag (with leading `v` stripped for parsing). Falls back to default branch HEAD if no tags exist.
  • Cache key is commit SHA (`//<sha[:12]>/`). Mutable branch refs re-resolve fresh on every fetch; cache hit only when SHAs match.
  • Atomic cache writes: fetch into `tempfile.mkdtemp`, then `os.replace` into the final dir. No partial-state visibility for concurrent runs.
  • Index and workflow always come from the same commit (`load_index(entry, ref=sha)`) — fixes a subtle bug where `latest` could resolve to a tag that doesn't contain a workflow listed in the default-branch index.
  • `registry list` / `registry show` drop the per-workflow "Versions" column and add a "Latest tags:" footer (up to 5 newest, with `, ...` if more) for github registries.
  • `registry update` help text updated to explain the SHA-based CDN bypass.

Implementation

ModuleChange
`registry/github.py`Added `get_default_branch`, `resolve_ref_to_sha`; `list_tags` now paginates (cap 1000)
`registry/version_resolver.py`New module: `resolve_ref` (latest tag → default branch fallback) and `materialize_to_sha`
`registry/index.py``load_index(entry, ref=None)` accepts an optional ref; github path always resolves to SHA before fetching
`registry/resolver.py`New `#`-based parser; `ResolvedRef.version` → `ResolvedRef.ref`
`registry/cache.py`Ref-aware, SHA-keyed, atomic fetch_workflow
`cli/registry.py`Drop Versions column, add Latest tags footer
`cli/app.py`Update 4 callers from `.version` → `.ref`

Tests

  • 153 registry tests pass (including 14 new tests in `test_version_resolver.py`).
  • Full suite: 2324 pass, 9 skipped, 0 regressions.
  • New coverage: tag pagination, semver sort with prereleases, mixed parseable/non-parseable tags, default-branch fallback, atomic-write failure cleanup, branch ref re-resolution, cache-hit skip, race-on-rename handling, path-registry-with-ref error.

Docs updated

  • `docs/design/registry.md` — full rewrite of Reference syntax, Versioning, and Caching sections; updated Implementation table.
  • `docs/cli-reference.md` — new syntax examples; `registry update`/`show`/`list` descriptions.
  • `README.md` — quick-start example uses `#ref` form.

Notes for reviewers

  • The `#` separator may need quoting in shells if surrounded by spaces (e.g. `conductor run 'qa-bot @ team # v1'`). Mid-token use (the typical case) is fine unquoted.
  • Branch refs are re-resolved to SHAs on every `fetch_workflow` call, which means an extra GitHub API roundtrip when running with a branch ref. Acceptable trade-off for correctness — explicit tag refs hit the cache directly.

Jason Robert (jrob5756)and others added 2 commits May 5, 2026 19:14
- New ref grammar: <workflow>[@<registry>][#<ref>] (was @@Version)
- Auto-discover versions from git tags; semver-sort with v-prefix strip
- Fall back to default branch HEAD when no tags exist
- Any tag/branch/SHA can be pinned via #ref
- Drop versions: field from index.yaml (silently ignored for back-compat)
- Cache by commit SHA (sha[:12]); mutable refs re-resolve fresh
- Atomic cache writes (tempfile.mkdtemp + os.replace)
- Index loads always resolve ref->SHA, fetching at /<sha>/index.yaml
to bypass raw.githubusercontent.com Fastly CDN
- registry list/show display 'Latest tags:' footer
- Path registries hard-error on non-empty refs
- Empty # and multiple @/# are hard errors
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address quality issues from comprehensive PR review:
- Declare packaging>=21.0 explicitly in pyproject.toml; remove the
silent ImportError fallback in version_resolver._sort_tags that
could return commit-date-ordered tags as 'latest'.
- Promote _sort_tags -> sort_tags (was imported across module
boundaries from cli/registry.py).
- Add RegistryNotFoundError(RegistryError) subclass; _raise_for_status
raises it on HTTP 404. _load_github_index now catches only
RegistryNotFoundError in the per-filename loop, so auth, rate-limit,
and network errors propagate clearly instead of being collapsed into
a misleading 'no index file found' message.
- _format_latest_tags now catches only RegistryError and httpx.HTTPError
(was bare Exception); surfaces the actual error message in the
'(unavailable: ...)' display.
- New prune_temp_dirs() helper cleans orphaned .tmp-* directories
(created by the atomic-write pattern) on registry update.
- Tighten misleading docstrings: list_tags clarifies 'commit-date order'
vs semver; fetch_workflow Raises documents silent sibling-fetch
failures.
10 new tests covering the new error propagation, tmp-dir cleanup,
and surfaced suggestions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756
Jason Robert (jrob5756) merged commit 33bc758 into mainMay 6, 2026
7 checks passed
@jrob5756
Jason Robert (jrob5756) deleted the feat/registry-tag-versioning branch May 6, 2026 00:56
Jason Robert (jrob5756) added a commit that referenced this pull request May 6, 2026
* chore: release 0.1.12
Bumps version to 0.1.12 and updates CHANGELOG with the four PRs
merged since v0.1.11:
- #149: Windows install diagnostics
- #151: Tag-based registry versioning with # ref syntax
- #152: Unified reasoning.effort configuration
- #153: Dashboard layout fix for human_gate options + loop-backs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: add #155 (Windows update reliability) to 0.1.12 changelog
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@jrob5756