Add system tests for Dataproc trigger on_kill cancel behavior by srchilukoori · Pull Request #65982 · apache/airflow

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori · 2026-04-27T22:39:15Z

Follow-up to #65742 as requested by @shahar1. Adds a system test DAG that exercises the cancel_on_kill behavior introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill().

Changes:

New file: providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py

The DAG contains two tests sharing a single Dataproc cluster:

Test	What it does	What it proves
A (happy path)	Deferrable submit with `cancel_on_kill=True` runs SparkPi to completion	Trigger plumbing with `cancel_on_kill=True` does not break normal deferrable execution
B (cancel path)	Submits a long-running SparkPi (1M slices) asynchronously, cancels it via `DataprocHook.cancel_job()`, asserts `CANCELLED` state	The `cancel_job()` call that `on_kill()` delegates to actually cancels running Dataproc jobs

Testing:

System tests require GCP credentials and a project with Dataproc API enabled. I do not have access to a GCP environment to run this test end-to-end. The test was verified via:

python3 -m py_compile — syntax OK
ruff check / ruff format — all checks passed
All pre-commit hooks passed
Pattern consistency verified against existing Dataproc system tests (example_dataproc_spark_deferrable.py, example_dataproc_spark_async.py, example_dataproc_start_from_trigger.py)

To run manually:

SYSTEM_TESTS_ENV_ID=<id> SYSTEM_TESTS_GCP_PROJECT=<project> \
pytest providers/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py -s

related: #65742

Was generative AI tooling used to co-author this PR?

Yes — Claude

shahar1

Great, thank you!

@VladaZakharova @MaksYermak - is it compatible for running on your environment? I'll be happy for your feedback.

VladaZakharova · 2026-04-28T07:14:49Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

shahar1 · 2026-04-28T08:45:59Z

hi
thank you for submitting PR.
In google provider we don't actually create a system test for every parameter we have, because then google provider will blow because of it's own size :)
I can see that you created this PR to test changes from another PR. I think in this case only good unit tests will be enough.

Is it possible to retain it, but exclude from the Google automated tests?

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

srchilukoori · 2026-04-29T14:06:36Z

Thanks @shahar1, made the change.

Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:

ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)

How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.

Verified locally with pytest --collect-only --system:

without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

shahar1 · 2026-04-29T15:14:26Z

Thanks @shahar1, made the change.
Added an env-var gate at module level in example_dataproc_cancel_on_kill.py:
ifnotos.environ.get("RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST"):
pytest.skip("Manual-only system test: set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 to run.", allow_module_level=True)
How this works: by default the test is skipped at collection time, so automated runs (which don't set the env var) won't pick it up. To run manually, set RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1. Same pytest.skip(..., allow_module_level=True) pattern already used for "service not available" gates in example_bigquery_to_mysql.py etc.
Verified locally with pytest --collect-only --system:
without the env var: 0 tests collected (SKIPPED with the explanation message)
with RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1: 1 test collected

Nice idea, but currently seems incompatible with other tests, as they fail - try to figure out how to work around that.

srchilukoori · 2026-04-29T21:24:17Z

Fixed in 678492e. Replaced pytest.skip(allow_module_level=True) with pytestmark = pytest.mark.skipif(...) — file imports cleanly now, test still skipped unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set.

cc: @VladaZakharova

@shahar1

Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state.

The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests.

Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure.

pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset.

potiuk · 2026-05-05T16:13:07Z

@srchilukoori — Your unresolved review thread(s) from @shahar1 appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the ready for maintainer review label so the PR re-enters the maintainer review queue.

@shahar1 — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @srchilukoori will follow up.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

shahar1 · 2026-05-05T16:39:17Z

Merged from main, good to merge after CI is green

srchilukoori · 2026-05-06T12:38:44Z

Merged from main, good to merge after CI is green

@shahar1 all checks are green. Please merge when you get a minute.

@shahar1

…#65982) * Add system tests for Dataproc trigger on_kill cancel behavior Follow-up to apache#65742 (requested by @shahar1). Adds a system test DAG that exercises the cancel_on_kill plumbing introduced when DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger migrated to BaseTrigger.on_kill(). Test A: deferrable submit with cancel_on_kill=True completes normally. Test B: submits a long-running job, cancels it via DataprocHook.cancel_job() (the same call on_kill delegates to), and asserts CANCELLED state. * Fix MyPy failure: use version-compat import pattern for task decorator The bare `from airflow.decorators import task` import fails MyPy in the providers check because `airflow.decorators` does not export `task` in Airflow 3.x. Switch to the standard `AIRFLOW_V_3_0_PLUS` conditional import pattern used by all other system tests. * Gate cancel_on_kill system test behind opt-in env var Per reviewer feedback (apache#65982), exclude the cancel_on_kill system test from automated Google CI runs. Test now skips at collection time unless RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST=1 is set, allowing manual end-to-end runs without burdening automated infrastructure. * Replace module-level pytest.skip with pytestmark skipif pytest.skip(allow_module_level=True) raises a Skipped exception at import time that DagBag records as an import error, breaking the test_should_be_importable parametrized test in DB-core CI jobs. pytestmark = pytest.mark.skipif(...) lets the file import cleanly (DagBag happy, importability test passes) while still skipping test_run when RUN_MANUAL_DATAPROC_CANCEL_ON_KILL_TEST is unset. * Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization --------- Co-authored-by: Sadha Chilukoori <ssreddy.8555@gmail.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

srchilukoori requested a review from shahar1 as a code owner April 27, 2026 22:39

boring-cyborgBot added area:providers provider:google Google (including GCP) related issues labels Apr 27, 2026

srchilukoori force-pushed the test/dataproc-trigger-on-kill-system-tests branch from a99a3c6 to f13986dCompare April 28, 2026 00:51

shahar1 approved these changes Apr 28, 2026
View reviewed changes

Sadha Chilukoori added 4 commits May 2, 2026 11:56

srchilukoori force-pushed the test/dataproc-trigger-on-kill-system-tests branch from 678492e to 649d397Compare May 2, 2026 18:56

shahar1 reviewed May 3, 2026
View reviewed changes

Comment threadproviders/google/tests/system/google/cloud/dataproc/example_dataproc_cancel_on_kill.py Outdated

Rename env flag to RUN_MANUAL_GOOGLE_SYSTEM_TESTS for generalization
d9f467c

potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 5, 2026

Merge branch 'main' into test/dataproc-trigger-on-kill-system-tests
7a34364

Merge branch 'main' into test/dataproc-trigger-on-kill-system-tests
55d3431

potiuk merged commit f38982c into apache:mainMay 10, 2026
94 checks passed

srchilukoori deleted the test/dataproc-trigger-on-kill-system-tests branch May 11, 2026 13:34