Implement execution_timeout semantics for AirbyteTriggerSyncOperator in deferrable mode

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.

[SameerMesiah97]

Description

This change enforces execution_timeout for AirbyteTriggerSyncOperator in deferrable mode.

Previously, when the operator deferred, execution_timeout was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.

The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.

Rationale

In non-deferrable mode, execution_timeout is enforced by the scheduler, which terminates the task process and invokes on_kill() to cancel the external Airbyte job.

In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, execution_timeout is a hard task-level limit and should behave consistently regardless of execution mode.

Without explicit handling in the trigger/operator interaction, execution_timeout becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.

This is the same class of issue addressed in PR #61472 for DbtCloudRunJobOperator.

Notes

• The existing timeout parameter only controls how long the operator waits for job completion and does not imply cancellation.
• When both execution_timeout and timeout are set, the earlier deadline takes precedence.
• In execute_complete, several generic AirflowException raises have been replaced with more appropriate, specific exception types.
• Cancellation on execution_timeout is implemented as best-effort. Failures during cancel_job are logged but do not mask the timeout; the task still fails with a timeout error.
• The same best-effort semantics have been implemented for on_kill for consistency; cancellation errors are logged and not re-raised.

Tests

• Added a trigger-level test asserting a timeout event when the execution deadline is exceeded.
• Added an operator-level test verifying job cancellation on execution timeout.
• Added an operator-level test asserting that cancellation failures are swallowed and do not mask the timeout (timeout is still raised).
• Added a operator-level test verifying that on_kill does not raise if cancel_job fails (best-effort behavior).
• Modified a trigger serialization test to ensure the optional execution_deadline field is always serialized.

Documentation

• Updated the trigger docstring to document the new execution_deadline parameter.
• Updated the operator docstring to clarify the behavior of timeout vs execution_timeout; added an entry for execution_timeout.

Backwards Compatibility

This change does not modify public APIs or method signatures.

Behavior is changed such that Airbyte jobs are now cancelled when execution_timeout is reached in deferrable mode. Previously, jobs could continue running after the task timed out.

Closes: #64048

[SameerMesiah97]

CI failure is unrelated to my changes. It was present in PR #64084, which only changes the spelling of a 'deferrable' in one of the log messages within the execute method of AirbyteTriggerSyncOperator.

[Copilot]

Pull request overview

Enforces execution_timeout semantics for AirbyteTriggerSyncOperator when running in deferrable mode by introducing an absolute execution_deadline passed to the trigger and handling timeout events by cancelling the Airbyte job.

Changes:

• Add execution_deadline support to AirbyteSyncTrigger (serialization + timeout event emission).
• Compute and pass an execution deadline from AirbyteTriggerSyncOperator in deferrable mode.
• Add/adjust unit tests for trigger timeout events and operator cancellation behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
providers/airbyte/src/airflow/providers/airbyte/triggers/airbyte.py	Add execution_deadline to trigger and emit a timeout event when exceeded.
providers/airbyte/src/airflow/providers/airbyte/operators/airbyte.py	Compute deadline before deferring; cancel Airbyte job on timeout event in execute_complete.
providers/airbyte/tests/unit/airbyte/triggers/test_airbyte.py	Add serialization coverage for execution_deadline and a trigger-level execution-timeout test.
providers/airbyte/tests/unit/airbyte/operators/test_airbyte.py	Add operator-level tests asserting cancellation + failure behavior on execution timeout.

[potiuk]

@SameerMesiah97 Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

• ❌ Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:airbyte,common.compat, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:airbyte,common.compat, Sqlite tests: providers / DB-prov:Sqlite:3.10:airbyte,common.compat. Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.

Note: Your branch is 327 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[SameerMesiah97]

@Subham-KRLX

Thank you for the review. I have addressed all your feedback where appropriate.

[SameerMesiah97]

@kaxil

I would appreciate another look once you get the time.

[SameerMesiah97]

Requesting review for this.