Uh oh!
There was an error while loading. Please reload this page.
ci: validate all reproducible codegen is up to date - #5553
Merged
Conversation
Replace the piecemeal generate-schema / generate-validation steps and the separate validate-python-codegen job with a single drift gate that runs the reproducible subset of `task generate` and asserts a clean tree. This is now possible because CLI command-stub generation is hermetic (driven by the checked-in .codegen/cli.json), so generate-cligen, generate-refschema and generate-direct can be checked alongside the existing schema/validation/pydabs generators. Adds a `generate-check` Taskfile target listing the generators that reproduce byte-for-byte on a clean checkout: cligen, refschema, schema, validation, direct, pydabs. It omits the three generators in `generate` whose output is not reproducible per-PR: - generate-tf-schema downloads the Terraform provider (no CI runner sets it up) - generate-schema-docs stamps x-since-version from git tags (published to the docgen branch by update-schema-docs.yml) - generate-docs stamps the current date via time.Now The job gains Python + uv (needed by generate-direct and pydabs-codegen) and uses `git add --intent-to-add` before `git diff` so newly generated files count, not just modifications. Co-authored-by: Isaac
The runner's pre-installed python3 already covers the python invocations generate-check needs (EMBED_SOURCES' list_embeds.py and the stdlib-only generate-direct-apitypes); the existing job relied on it too. uv covers the remaining uv-run generators (pydabs-codegen, generate-direct-resources). EMBED_SOURCES keeps a python3 dependency repo-wide, so unifying everything on uv isn't possible here. Co-authored-by: Isaac
Rename validate-generated-is-up-to-date -> validate-generated. Switch to setup-build-environment (with a dedicated cache-key so it doesn't evict the primary test caches) for Go + Python + uv, and add `go mod download` before generate-check. generate-cligen formats the regenerated stubs with goimports, which adds imports it can only resolve from the module cache (e.g. cobra in cmd/workspace/cmd.go, whose template doesn't emit it). On a cold cache goimports silently omits the import and the regenerated tree fails to build. Co-authored-by: Isaac
janniklasrose
approved these changes
Jun 11, 2026
janniklasrose
left a comment
Contributor
There was a problem hiding this comment.
Finally validate-generated-is-up-to-date pass is complete/no longer giving false sense of security
Uh oh!
There was an error while loading. Please reload this page.
setup-jfrog (via setup-build-environment) points uv at the JFrog pypi proxy, so pydabs-codegen's `uv run` rewrites python/uv.lock and python/codegen/uv.lock with proxy URLs (and drops some size fields). The committed locks use pypi.org, so that is environment noise rather than codegen drift; restore the files before the git diff check. Co-authored-by: Isaac
Per review (janniklasrose): the gate also checks non-code artifacts like jsonschema.json and the *.yml files, so "files" is more accurate than "code". Co-authored-by: Isaac
eng-dev-ecosystem-bot
commented
Jun 11, 2026
Collaborator
Commit: 2d3088c
24 interesting tests: 15 SKIP, 7 KNOWN, 2 flaky
Top 28 slowest tests (at least 2 minutes):
|
eng-dev-ecosystem-bot
commented
Jun 11, 2026
Collaborator
Commit: f63abcd
475 interesting tests: 394 MISS, 37 FAIL, 25 RECOVERED, 13 KNOWN, 4 PANIC, 2 SKIP
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Now that CLI command-stub generation is hermetic, fold the per-generator
generate-schema/generate-validationsteps and thevalidate-python-codegenjob into one gate that runs the reproducible subset oftask generateand asserts a clean tree.The new
generate-checktask runs the generators that reproduce byte-for-byte on a clean checkout (cligen, refschema, schema, validation, direct, pydabs). It omits the three ingeneratethat can't reproduce per-PR, documented inline:generate-tf-schema(downloads the TF provider),generate-schema-docs(git-tagx-since-version, published todocgen), andgenerate-docs(stampstime.Now).This pull request and its description were written by Isaac.