Uh oh!
There was an error while loading. Please reload this page.
fix: [AI-7675] accept reused/unknown run statuses in run_results v1-v5 parsers - #108
Conversation
…-v5 parsers Mirror of the v6 `Status` shim (AI-7435, PR #106) for the pre-v6 schemas, closing the residual of AI-7675 finding #2: any run status outside `success`/`error`/`skipped` (e.g. dbt 2.0 `reused`, or any future status) raised a `ValidationError` in run_results v1-v5 and the ingestion worker silently dropped the entire `run_results.json`. - Make `Status` a `str, Enum`, add `reused`, and add the `_missing_` forward-compat fallback so unknown statuses surface as real members instead of failing validation (identical to the v6 fix). - Reorder `RunResultOutput.status` to `Union[Status1, Status2, Status]` so test/freshness statuses (`pass`/`fail`/`warn`/`runtime error`) keep resolving via the strict enums and the permissive run `Status` is tried last (same reordering as v6). - Add parameterized regression tests over v1-v5 mirroring `test_run_results_v6.py`, including full-file `parse_run_results`. Freshness-specific enums (`Status3`/`Status4` in v3/v4) are untouched, matching the v6 fix scope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Files Reviewed (6 files)
The Reviewed by glm-5.2 · Input: 35.7K · Output: 8.4K · Cached: 432.3K |
Uh oh!
There was an error while loading. Please reload this page.
…#110) `tests/test_vendor/test_run_results_pre_v6.py` (PR #108) was not formatted to the repo's `black` line-length (140), which fails the `pre-commit` gate inside `release.sh`/`tox` and blocked the v0.3.4 PyPI publish in the tag-release workflow. Formatting-only change; tests unchanged semantically (36 pass). This PR carries the `version-bump` label so its merge re-triggers tag-release and publishes 0.3.4 (the previous unpublished v0.3.4 tag is deleted so the workflow can recreate it at this commit). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…sources v1-v3 (#114) The dbt Fusion engine serializes source-freshness `status` with its Rust variant names -- `"Pass"` / `"Warn"` / `"Error"` -- while dbt-core and the published `sources/v3.json` schema Fusion itself stamps into the artifact both use the lowercase forms. Every result row therefore failed both members of the `results` union, the whole `sources.json` raised a `ValidationError`, and the ingestion worker silently dropped it. Nine of harvestgroup's ten production environments have zero source-freshness rows as a result. This is the residual of the AI-7675 work: PR #106/#108 added the `_missing_` forward-compat shim to the `run_results` status enums but explicitly left the freshness enums, and the `sources` parsers entirely, untouched. - Make `Status1` a `str, Enum` whose `_missing_` case-folds to the canonical lowercase member first, then falls back to the same forward-compat pseudo-member used by the `run_results` shim for unknown statuses. - Case-folding rather than adding PascalCase members is deliberate: the extractor persists `status.value`, and every dbt-core-backed tenant already writes lowercase into the same table. - Leave the runtime-error-only `Status` enum strict. Fusion has no runtime-error variant, and loosening it would let a row missing a required field fall silently into the field-less branch instead of erroring. - Apply to v1-v3 rather than v3 alone, mirroring how #108 had to follow #106 across the older schemas. Verified against 432 real production artifacts spanning all nine Fusion environments and 13 Fusion builds (preview.190 -> .210), run through parse AND the worker's own `extract_sources`: 432/432 parsed, 10,893 freshness rows extracted, every capitalized input landing as its lowercase counterpart. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
Closes the residual of AI-7675 finding #2: the
reused-status shim shipped for run_results v6 in 0.3.2 (AI-7435, PR #106) was never applied to v1–v5. Any run status outsidesuccess/error/skippedstill raises aValidationErroron those schema versions, and the backend ingestion worker silently drops the entirerun_results.json(batch stays green — same silent-drop failure mode as AI-7435/AI-7856).Changes
Exact mirror of the v6 fix, applied to
run_results_v{1..5}.py:Statusbecomesstr, Enum, gainsreused, and gains the_missing_forward-compat fallback — unknown future dbt statuses parse as real members instead of failing validation.RunResultOutput.statusunion reordered toUnion[Status1, Status2, Status]so test/freshness statuses (pass/fail/warn/runtime error) keep resolving via the strict enums; the permissive runStatusis tried last.Status3/Status4in v3/v4) untouched — matching the v6 fix scope.Tests
New
tests/test_vendor/test_run_results_pre_v6.py, parameterized over v1–v5, mirroringtest_run_results_v6.py:Also verified against a real sunrun v4 production artifact (
run_498571806from S3): parses toRunResultsV4with all results intact. Adjacent consumers (tests/clients/altimate/test_utils.py,tests/core/platform/dbt/test_artifact_loaders.py) pass.ruffclean on the new test; zero new violations in the generated vendor files (pre-existing UP007 counts unchanged).Scope notes
🤖 Generated with Claude Code