Download task logs by collinmcnulty · Pull Request #54481 · apache/airflow

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

collinmcnulty · 2025-08-13T19:54:10Z

Added ability to download task instance logs in airflow 3 ui as described in issue #47689
closes: #47689

Duplicates and rebases #49412 which appears abandoned

…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

Add path existence check to bundle initialization to help users identify configuration issues early. When a bundle is initialized but its path doesn't exist on disk, Airflow now logs a clear warning message indicating the missing path and potential DAG loading issues. This improves the debugging experience for users setting up DAG bundles by providing immediate feedback when bundle paths are misconfigured.

* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

…he#53990) Enhanced error handling in the AWS Lambda executor to better distinguish between Lambda service failures (DLQ) and task execution failures. Added check to properly identify DLQ failures and updated error messages to clearly indicate whether the failure was at the Lambda service level (timeout/crash/memory) or during task execution.

…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

…he#53995) This will land in 3.0.4, not 3.1.0.

…pache#54009)

…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

* fix typo in http conn docs * fix typo in conn docs

) When we reach timeut we kill all the hanging containers already and after the pool has been terminated, we will print all the logs. However, when the pool had not yet been fully executing (i.e the containers were hanging and some tasks were not started) - without terminating the pool that would kill running containers and the remaining tasks would start new ones. This PR changes the timeout handler to terminate the pool before attempting to kill all the containers. It also turned out that exit handling by the main thread monitorint the tests in this case would hang rather than print logs: * it was waiting in a loop to wait for all task to complete (which would never happen) * it was trying to retrieve result from ApplyResult without timeout where it would hang for ever for terminated tasks This PR introduces a separate path to handle timeout, which does not wait for those two and handles timeout immediately. It also refactors the whole "end of tests" method splitting it into several methods to make it easier to reason and read.

…pache#53126)

) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

pierrejeambrunand others added 30 commits August 13, 2025 14:48

Handle unmodified sensitived fields when updating connections (apache…
fe5ae41
…#53943) * Fix redacted values editing * Small improvements * Small adjustments * Update UI and fix some errors * Address PR comments

FigConfigmenu item missing inFabAuthManager(apache#53944)
6cd766e

Additional usages of timezone in AWS Provider (apache#53986)
96362fe

Add type annotation for dev system tests (apache#53979)
abc922c

Fix elements nesting warning in DataTable (apache#53980)
3e21fb7

Fix nth-child warning (apache#53982)
7af97b0

fix(i18n): Namespace i18n keys for Backfills view (apache#53992)
6111ecc

Remove parameter from Edge example (apache#53997)
5b7df07

i18n(Ko): add missing translations (apache#53978)
001d3c3
* i18n(Ko): add missing translations * Fix translation for "Dag Run ID" to "Dag 실행 ID"

Allow overriding schedulerName on worker/tasks pods (apache#53983)
2b7afb0

feat(hitl): add HITLBranchOperator (apache#53960)
adb2ac4

Fix dag_versions property when created_dag_version is None with bundl…
cf780e2
…e_version set (apache#53952) Fix dag_versions property when created_dag_version is None with bundle_version set

Add ALL_DONE_MIN_ONE_SUCCESS trigger rule (apache#53959)
c3cd25c

Fix version forparsing_pre_import_modulesconfig option move (apac…
d29810e
…he#53995) This will land in 3.0.4, not 3.1.0.

build(pre-commit): upgrade node version to 22.18.0 and flynt 1.0.6 (a…
165d264
…pache#54009)

Fix AIRFLOW_API_APPS constant in api_server_command (apache#54007)
e718877

Fix the google cloud provider RayHook protobuf compatibility (apache#…
d68d14a
…54014) - The previously used `google._upb._message.ScalarMapContainer` is not available in protobuf >= 5.*, which breaks RayHook and operators. Co-authored-by: Oleg Kachur <kachur@google.com>

Conn docs extra fix (apache#53942)
56c7b87
* fix typo in http conn docs * fix typo in conn docs

Add support for taking the config file as a dict for KubernetesHook (a…
6ea23fd
…pache#53126)

Fix missing prefetch arg to SFTPOperator (apache#53906)
0752fc6

Fix flaky test test_alias (apache#54001)
d1f7f11

fix: Add FAB configs for cookie security (apache#53542)
a3e8f1a

Fix toaster messages (apache#54019)
46e20a2

i18n(tw): Add missing translations (apache#53991)
a80a3ef

unmount dialog & add onOpen (apache#53307)
4803629

Fill translation gap in PL - 19.07.2025 (apache#53528)
0180e77

Increase zIndex for tooltip visibility in GridTI component (apache#53913
95d426d
) * Increase zIndex for tooltip visibility in GridTI component * Change zIndex value to Chakra token

boring-cyborgBot added area:dev-tools backport-to-v3-0-test labels Aug 13, 2025

collinmcnulty closed this Aug 13, 2025

pierrejeambrun mentioned this pull request Oct 17, 2025
UI - Download Tasks Logs button #56771
Merged