Prevent Triggerer from crashing when a trigger event isn't serializable by dabla · Pull Request #60152 · apache/airflow

dabla · 2026-01-06T08:00:48Z

This PR is related to the issue #60120 in which the trigger process crashes if a returned trigger event isn't serializable. This PR will catch the NotImpelmentedError and detect which event caused the crash and cancel the trigger associated to that event that caused the error.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

…mplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger

…y happy

potiuk · 2026-01-08T13:16:12Z

Looks good and seems it would be great to have it in 3.1.6 - but I would love someone with more triggerer expertise reviews it.

github-actions · 2026-01-22T18:22:22Z

Backport failed to create: v3-1-test. View the failure log Run details

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker 0238244 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

…le (#60152) * refactor: If asend in TriggerRunner comms decoder crashes due to NotImplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger * refactor: Applied some reformatting * refactor: Fixed some mypy issues * refactor: Fixed return type of send_changes method * refactor: Changed imports of trigger events * refactor: Reformatted trigger job runner * refactor: Fixed mocking of comms decoder * refactor: Forgot to add the patched supervisor_builder * refactor: Changed asserts in test_sync_state_to_supervisor * refactor: Refactored how state changes are validated * refactor: Validate events while creating the TriggerStateChanges message * refactor: Refactored try/except in validate_state_changes to keep mypy happy * Update airflow-core/src/airflow/jobs/triggerer_job_runner.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> * refactor: Renamed validate_state_changes method to process_trigger_events * refactor: Only sanitize invalid trigger events if first attempt fails * refactor: Should check if msg.events is not None * refactor: Adapted test_sync_state_to_supervisor so both initial and retry call are correctly asserted and bad events are being sanitized --------- Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> (cherry picked from commit 0238244)

dabla · 2026-01-22T19:52:20Z

Backported to v3-1-test

pierrejeambrun · 2026-01-23T12:15:49Z

@dabla how did you backport this? I think it's breaking the CI and I can't find the related PR, I'm surprised this could be merged on v3-1-test without static check succeeding.

I have opened a revert PR there #60976 to see if the CI is happy.

…rializable (apache#60152)" This reverts commit b7d1c41.

…rializable (#60152)" (#60976) This reverts commit b7d1c41.

dabla · 2026-01-23T15:29:06Z

@dabla how did you backport this? I think it's breaking the CI and I can't find the related PR, I'm surprised this could be merged on v3-1-test without static check succeeding.
https://github.com/apache/airflow/actions/runs/21285259040/job/61265554224?pr=60958
https://github.com/apache/airflow/actions/runs/21285249743/job/61265446226?pr=60959
I have opened a revert PR there #60976 to see if the CI is happy.

Salut @pierrejeambrun. Well I did as was described above, I checked out the v3-1-test, cherry-picked the commit from this PR, resolved merged conflicts, and pushed it. Will create branch from v3-1-test and create PR from there, thx for reverting it.

…le (#60152) * refactor: If asend in TriggerRunner comms decoder crashes due to NotImplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger * refactor: Applied some reformatting * refactor: Fixed some mypy issues * refactor: Fixed return type of send_changes method * refactor: Changed imports of trigger events * refactor: Reformatted trigger job runner * refactor: Fixed mocking of comms decoder * refactor: Forgot to add the patched supervisor_builder * refactor: Changed asserts in test_sync_state_to_supervisor * refactor: Refactored how state changes are validated * refactor: Validate events while creating the TriggerStateChanges message * refactor: Refactored try/except in validate_state_changes to keep mypy happy * Update airflow-core/src/airflow/jobs/triggerer_job_runner.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> * refactor: Renamed validate_state_changes method to process_trigger_events * refactor: Only sanitize invalid trigger events if first attempt fails * refactor: Should check if msg.events is not None * refactor: Adapted test_sync_state_to_supervisor so both initial and retry call are correctly asserted and bad events are being sanitized --------- Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> (cherry picked from commit 0238244)

dabla · 2026-01-23T15:48:40Z

@pierrejeambrun I've create PR #60981 for the backport. Thx to @shahar1 for helping me out.

…le (#60152) * refactor: If asend in TriggerRunner comms decoder crashes due to NotImplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger * refactor: Applied some reformatting * refactor: Fixed some mypy issues * refactor: Fixed return type of send_changes method * refactor: Changed imports of trigger events * refactor: Reformatted trigger job runner * refactor: Fixed mocking of comms decoder * refactor: Forgot to add the patched supervisor_builder * refactor: Changed asserts in test_sync_state_to_supervisor * refactor: Refactored how state changes are validated * refactor: Validate events while creating the TriggerStateChanges message * refactor: Refactored try/except in validate_state_changes to keep mypy happy * Update airflow-core/src/airflow/jobs/triggerer_job_runner.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> * refactor: Renamed validate_state_changes method to process_trigger_events * refactor: Only sanitize invalid trigger events if first attempt fails * refactor: Should check if msg.events is not None * refactor: Adapted test_sync_state_to_supervisor so both initial and retry call are correctly asserted and bad events are being sanitized --------- Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> (cherry picked from commit 0238244)

…le (#60152) (#60981) * refactor: If asend in TriggerRunner comms decoder crashes due to NotImplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger * refactor: Applied some reformatting * refactor: Fixed some mypy issues * refactor: Fixed return type of send_changes method * refactor: Changed imports of trigger events * refactor: Reformatted trigger job runner * refactor: Fixed mocking of comms decoder * refactor: Forgot to add the patched supervisor_builder * refactor: Changed asserts in test_sync_state_to_supervisor * refactor: Refactored how state changes are validated * refactor: Validate events while creating the TriggerStateChanges message * refactor: Refactored try/except in validate_state_changes to keep mypy happy * Update airflow-core/src/airflow/jobs/triggerer_job_runner.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> * refactor: Renamed validate_state_changes method to process_trigger_events * refactor: Only sanitize invalid trigger events if first attempt fails * refactor: Should check if msg.events is not None * refactor: Adapted test_sync_state_to_supervisor so both initial and retry call are correctly asserted and bad events are being sanitized --------- Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> (cherry picked from commit 0238244)

dabla · 2026-01-24T19:10:10Z

PR Merged into v3-1-test.

…le (apache#60152) * refactor: If asend in TriggerRunner comms decoder crashes due to NotImplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger * refactor: Applied some reformatting * refactor: Fixed some mypy issues * refactor: Fixed return type of send_changes method * refactor: Changed imports of trigger events * refactor: Reformatted trigger job runner * refactor: Fixed mocking of comms decoder * refactor: Forgot to add the patched supervisor_builder * refactor: Changed asserts in test_sync_state_to_supervisor * refactor: Refactored how state changes are validated * refactor: Validate events while creating the TriggerStateChanges message * refactor: Refactored try/except in validate_state_changes to keep mypy happy * Update airflow-core/src/airflow/jobs/triggerer_job_runner.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> * refactor: Renamed validate_state_changes method to process_trigger_events * refactor: Only sanitize invalid trigger events if first attempt fails * refactor: Should check if msg.events is not None * refactor: Adapted test_sync_state_to_supervisor so both initial and retry call are correctly asserted and bad events are being sanitized --------- Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

refactor: If asend in TriggerRunner comms decoder crashes due to NotI…
082fc70
…mplementedError as a trigger event is not serializable, then retry without that event and cancel associated trigger

dabla requested review from dstandish and hussein-awala as code owners January 6, 2026 08:00

boring-cyborgBot added the area:Triggerer label Jan 6, 2026

Merge branch 'main' into fix/trigger-crash-serialization-comms
5ca3b39

dabla marked this pull request as draft January 6, 2026 08:12

dabla added 20 commits January 6, 2026 09:33

refactor: Applied some reformatting
b303403

refactor: Fixed some mypy issues
5de68c1

refactor: Fixed return type of send_changes method
14082fe

refactor: Changed imports of trigger events
9cbf5a8

Merge branch 'main' into fix/trigger-crash-serialization-comms
3045fad

Merge branch 'main' into fix/trigger-crash-serialization-comms
6ebf6a5

refactor: Reformatted trigger job runner
e9dc8a4

Merge branch 'main' into fix/trigger-crash-serialization-comms
65999fe

refactor: Fixed mocking of comms decoder
6d70c53

Merge branch 'main' into fix/trigger-crash-serialization-comms
693415c

refactor: Forgot to add the patched supervisor_builder
a9a6cd8

refactor: Changed asserts in test_sync_state_to_supervisor
dc581f9

Merge branch 'main' into fix/trigger-crash-serialization-comms
521aac4

Merge branch 'main' into fix/trigger-crash-serialization-comms
5b98350

refactor: Refactored how state changes are validated
60421a0

refactor: Validate events while creating the TriggerStateChanges message
60bc489

Merge branch 'main' into fix/trigger-crash-serialization-comms
fc627b0

refactor: Refactored try/except in validate_state_changes to keep myp…
a7ffaaa
…y happy

Merge branch 'main' into fix/trigger-crash-serialization-comms
0b90b78

Merge branch 'main' into fix/trigger-crash-serialization-comms
b29e947

dabla marked this pull request as ready for review January 8, 2026 13:09

potiuk approved these changes Jan 8, 2026
View reviewed changes

potiuk added this to the Airflow 3.1.6 milestone Jan 8, 2026

dabla requested a review from ashb January 19, 2026 21:25

potiuk approved these changes Jan 20, 2026
View reviewed changes

potiuk reviewed Jan 20, 2026
View reviewed changes

Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py

dabla merged commit 0238244 into apache:mainJan 22, 2026
71 checks passed

dabla mentioned this pull request Jan 22, 2026
Triggerer crashes when a SQLExecuteQueryTrigger returns a non serializable result #60120
Closed
2 tasks

pierrejeambrun added a commit to astronomer/airflow that referenced this pull request Jan 23, 2026
Revert "Prevent Triggerer from crashing when a trigger event isn't se…
facee8a
…rializable (apache#60152)" This reverts commit b7d1c41.

pierrejeambrun mentioned this pull request Jan 23, 2026
Revert "Prevent Triggerer from crashing when a trigger event isn't se… #60976
Merged
1 task

pierrejeambrun added a commit that referenced this pull request Jan 23, 2026
Revert "Prevent Triggerer from crashing when a trigger event isn't se…
937f445
…rializable (#60152)" (#60976) This reverts commit b7d1c41.

dabla mentioned this pull request Jan 23, 2026
Backport PR #60152: Prevent Triggerer from crashing when a trigger event isn't serializable #60980
Closed
1 task

ephraimbuddy added type:bug-fix Changelog: Bug Fixes changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) and removed type:bug-fix Changelog: Bug Fixes labels Jan 27, 2026

dabla mentioned this pull request Jan 28, 2026
Implement specialized get_first and get_records method in OracleHook #61144
Merged
1 task

Navigation Menu

Uh oh!

Prevent Triggerer from crashing when a trigger event isn't serializable - #60152

Prevent Triggerer from crashing when a trigger event isn't serializable#60152
dabla merged 31 commits into
apache:mainfrom
dabla:fix/trigger-crash-serialization-comms

dabla commented Jan 6, 2026

Uh oh!

potiuk commented Jan 8, 2026

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Jan 22, 2026

Uh oh!

dabla commented Jan 22, 2026

Uh oh!

pierrejeambrun commented Jan 23, 2026•
edited
Loading

Uh oh!

dabla commented Jan 23, 2026•
edited
Loading

Uh oh!

dabla commented Jan 23, 2026•
edited
Loading

Uh oh!

dabla commented Jan 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

dabla commented Jan 6, 2026

Uh oh!

potiuk commented Jan 8, 2026

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Jan 22, 2026

Backport failed to create: v3-1-test. View the failure log Run details

Uh oh!

dabla commented Jan 22, 2026

Uh oh!

pierrejeambrun commented Jan 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dabla commented Jan 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dabla commented Jan 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dabla commented Jan 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pierrejeambrun commented Jan 23, 2026•
edited
Loading

dabla commented Jan 23, 2026•
edited
Loading

dabla commented Jan 23, 2026•
edited
Loading