Uh oh!
There was an error while loading. Please reload this page.
feat: Expose error classifier - #2539
Conversation
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
murarustefaan
left a comment
There was a problem hiding this comment.
I think this is good overall, but requesting a review from @erezrokah as well before merging.
There was a problem hiding this comment.
Pull request overview
This PR introduces a plugin-configurable error classification hook so resolver errors can be suppressed at the scheduler/plugin level (e.g., ignoring expected 404s) rather than handled in each resolver.
Changes:
- Added
schema.ErrorClassifier(+ supportingErrorEvent/ErrorPhase) and threaded it through scheduler/resolver execution paths. - Introduced
scheduler.WithErrorClassifier(...)and wired the classifier into DFS + shuffle-queue worker execution and resource-chunk resolution. - Added tests verifying default behavior (raised) vs suppressed behavior across strategies and resolver phases.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| schema/error_classifier.go | Adds the new public error-classifier API types and nil-safe helper. |
| scheduler/scheduler.go | Re-exports schema classifier types/phases and adds WithErrorClassifier. |
| scheduler/scheduler_shuffle_queue.go | Passes classifier into the shuffle-queue scheduler. |
| scheduler/scheduler_dfs.go | Consults classifier for table-resolver errors and passes it into chunk resolution. |
| scheduler/resolvers/resolvers.go | Adds classifier plumbing for chunk/column/pre/post resolver error handling. |
| scheduler/queue/scheduler.go | Adds option + field to carry classifier into queue workers. |
| scheduler/queue/worker.go | Consults classifier for table-resolver errors and passes it into chunk resolution. |
| scheduler/scheduler_error_classifier_test.go | Verifies scheduler-level suppression vs default SyncError emission across strategies. |
| scheduler/resolvers/resolvers_test.go | Updates existing resolver tests for new function signature. |
| scheduler/resolvers/error_classifier_test.go | Adds phase coverage tests ensuring suppression affects metrics and event metadata. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
🤖 I have created a release *beep* *boop* --- ## [4.96.0](v4.95.3...v4.96.0) (2026-08-03) ### Features * Expose error classifier ([#2539](#2539)) ([e042d73](e042d73)) ### Bug Fixes * **deps:** Update aws-sdk-go-v2 monorepo ([#2521](#2521)) ([cc5ce53](cc5ce53)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2524](#2524)) ([d080fbf](d080fbf)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2537](#2537)) ([f5b5033](f5b5033)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2540](#2540)) ([579a722](579a722)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2542](#2542)) ([83ceb77](83ceb77)) * **deps:** Update aws-sdk-go-v2 monorepo ([#2548](#2548)) ([6be63a3](6be63a3)) * **deps:** Update github.com/cloudquery/jsonschema digest to 45e7e20 ([#2535](#2535)) ([acb4b1b](acb4b1b)) * **deps:** Update go module directive to v1.26.4 ([#2528](#2528)) ([66974c9](66974c9)) * **deps:** Update go module directive to v1.26.5 ([#2547](#2547)) ([cb54f5d](cb54f5d)) * **deps:** Update golang.org/x/exp digest to c48552f ([#2527](#2527)) ([4b94b83](4b94b83)) * **deps:** Update module github.com/apache/arrow-go/v18 to v18.7.0 ([#2544](#2544)) ([301cb8c](301cb8c)) * **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.14.12 ([#2516](#2516)) ([2107281](2107281)) * **deps:** Update module github.com/getsentry/sentry-go to v0.47.0 ([#2529](#2529)) ([94da8e3](94da8e3)) * **deps:** Update module github.com/getsentry/sentry-go to v0.48.0 ([#2549](#2549)) ([133be1f](133be1f)) * **deps:** Update module golang.org/x/text to v0.40.0 ([#2550](#2550)) ([dc3e10e](dc3e10e)) * **deps:** Update module google.golang.org/grpc to v1.82.0 ([#2538](#2538)) ([aee8e3f](aee8e3f)) * **deps:** Update module google.golang.org/grpc to v1.82.1 ([#2543](#2543)) ([7182a82](7182a82)) * **deps:** Update opentelemetry-go monorepo ([#2532](#2532)) ([90f446e](90f446e)) * Preserve int64 precision when rebuilding nested values from strings ([#2553](#2553)) ([8599fa0](8599fa0)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This PR exposes a way for plugins to capture and classify errors at a plugin level rather than having to handle it on a per resolver basis.
For example this can be utilized in our AWS plugin to handle 404s or AWS specific errors without having to replace or wrap the error handling in every single table resolver
Example PR that uses this functionality: https://github.com/cloudquery/cloudquery-private/pull/13197