Skip to content

feat: Expose error classifier - #2539

Merged
kodiakhq[bot] merged 4 commits into
mainfrom
expose-error-classifier
Jul 16, 2026
Merged

feat: Expose error classifier#2539
kodiakhq[bot] merged 4 commits into
mainfrom
expose-error-classifier

Conversation

@bbernays

@bbernaysbbernays commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

This PR exposes a way for plugins to capture and classify errors at a plugin level rather than having to handle it on a per resolver basis.

For example this can be utilized in our AWS plugin to handle 404s or AWS specific errors without having to replace or wrap the error handling in every single table resolver

Example PR that uses this functionality: https://github.com/cloudquery/cloudquery-private/pull/13197

@bbernays
bbernays requested a review from a team as a code ownerJuly 15, 2026 16:01
@bbernays
bbernays requested a review from disqJuly 15, 2026 16:01
Comment threadscheduler/queue/worker.go Outdated
Comment threadscheduler/resolvers/resolvers.go Outdated

@murarustefaanmurarustefaan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good overall, but requesting a review from @erezrokah as well before merging.

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a plugin-configurable error classification hook so resolver errors can be suppressed at the scheduler/plugin level (e.g., ignoring expected 404s) rather than handled in each resolver.

Changes:

  • Added schema.ErrorClassifier (+ supporting ErrorEvent/ErrorPhase) and threaded it through scheduler/resolver execution paths.
  • Introduced scheduler.WithErrorClassifier(...) and wired the classifier into DFS + shuffle-queue worker execution and resource-chunk resolution.
  • Added tests verifying default behavior (raised) vs suppressed behavior across strategies and resolver phases.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
FileDescription
schema/error_classifier.goAdds the new public error-classifier API types and nil-safe helper.
scheduler/scheduler.goRe-exports schema classifier types/phases and adds WithErrorClassifier.
scheduler/scheduler_shuffle_queue.goPasses classifier into the shuffle-queue scheduler.
scheduler/scheduler_dfs.goConsults classifier for table-resolver errors and passes it into chunk resolution.
scheduler/resolvers/resolvers.goAdds classifier plumbing for chunk/column/pre/post resolver error handling.
scheduler/queue/scheduler.goAdds option + field to carry classifier into queue workers.
scheduler/queue/worker.goConsults classifier for table-resolver errors and passes it into chunk resolution.
scheduler/scheduler_error_classifier_test.goVerifies scheduler-level suppression vs default SyncError emission across strategies.
scheduler/resolvers/resolvers_test.goUpdates existing resolver tests for new function signature.
scheduler/resolvers/error_classifier_test.goAdds phase coverage tests ensuring suppression affects metrics and event metadata.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadschema/error_classifier.go
Comment threadscheduler/resolvers/resolvers.go Outdated
@kodiakhq
kodiakhqBot merged commit e042d73 into mainJul 16, 2026
10 checks passed
@kodiakhq
kodiakhqBot deleted the expose-error-classifier branch July 16, 2026 14:01
@cloudquery-cicloudquery-ciBot mentioned this pull request Jul 16, 2026
kodiakhqBot pushed a commit that referenced this pull request Aug 3, 2026
🤖 I have created a release *beep* *boop*
---
## [4.96.0](v4.95.3...v4.96.0) (2026-08-03)
### Features
* Expose error classifier ([#2539](#2539)) ([e042d73](e042d73))
### Bug Fixes
* **deps:** Update aws-sdk-go-v2 monorepo ([#2521](#2521)) ([cc5ce53](cc5ce53))
* **deps:** Update aws-sdk-go-v2 monorepo ([#2524](#2524)) ([d080fbf](d080fbf))
* **deps:** Update aws-sdk-go-v2 monorepo ([#2537](#2537)) ([f5b5033](f5b5033))
* **deps:** Update aws-sdk-go-v2 monorepo ([#2540](#2540)) ([579a722](579a722))
* **deps:** Update aws-sdk-go-v2 monorepo ([#2542](#2542)) ([83ceb77](83ceb77))
* **deps:** Update aws-sdk-go-v2 monorepo ([#2548](#2548)) ([6be63a3](6be63a3))
* **deps:** Update github.com/cloudquery/jsonschema digest to 45e7e20 ([#2535](#2535)) ([acb4b1b](acb4b1b))
* **deps:** Update go module directive to v1.26.4 ([#2528](#2528)) ([66974c9](66974c9))
* **deps:** Update go module directive to v1.26.5 ([#2547](#2547)) ([cb54f5d](cb54f5d))
* **deps:** Update golang.org/x/exp digest to c48552f ([#2527](#2527)) ([4b94b83](4b94b83))
* **deps:** Update module github.com/apache/arrow-go/v18 to v18.7.0 ([#2544](#2544)) ([301cb8c](301cb8c))
* **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.14.12 ([#2516](#2516)) ([2107281](2107281))
* **deps:** Update module github.com/getsentry/sentry-go to v0.47.0 ([#2529](#2529)) ([94da8e3](94da8e3))
* **deps:** Update module github.com/getsentry/sentry-go to v0.48.0 ([#2549](#2549)) ([133be1f](133be1f))
* **deps:** Update module golang.org/x/text to v0.40.0 ([#2550](#2550)) ([dc3e10e](dc3e10e))
* **deps:** Update module google.golang.org/grpc to v1.82.0 ([#2538](#2538)) ([aee8e3f](aee8e3f))
* **deps:** Update module google.golang.org/grpc to v1.82.1 ([#2543](#2543)) ([7182a82](7182a82))
* **deps:** Update opentelemetry-go monorepo ([#2532](#2532)) ([90f446e](90f446e))
* Preserve int64 precision when rebuilding nested values from strings ([#2553](#2553)) ([8599fa0](8599fa0))
---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@bbernays@murarustefaan@erezrokah