Skip to content

direct: treat tasks as a map identified by task_key - #4029

Merged
denik merged 4 commits into
mainfrom
denik/task_key_order
Dec 1, 2025
Merged

direct: treat tasks as a map identified by task_key#4029
denik merged 4 commits into
mainfrom
denik/task_key_order

Conversation

@denik

@denikdenik commented Nov 28, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Changes

Configure "task_key" as a way to identify tasks in jobs.

Builds on top of #4014

Why

Makes diffs more precise. Ignores reordering of slice without need of sorting (#3212). Note, we still sort, this is to be reverted in separate PR.

Tests

Existing tests. New test that reorders tasks locally.

@denikdenik changed the title direct: treat tasks as a mapdirect: treat tasks as a map identified by task_keyNov 28, 2025
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 28, 2025•
edited
Loading

Copy link
Copy Markdown
Collaborator

Commit: 27d32a9

Run: 19765325928

Env🟨​KNOWN🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
🟨​aws linux75236662919:26
🟨​aws windows72237162722:24
💚​aws-ucws linux7251451423:13
💚​aws-ucws windows7251651223:23
🔄​azure linux41436762819:41
🔄​azure windows21437162618:02
🔄​azure-ucws linux2450951324:29
💚​azure-ucws windows1451251123:05
🔄​gcp linux11436363219:01
🔄​gcp windows3436463016:34
26 failing tests:
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
🟨​TestAccept🟨​K🟨​K💚​R💚​R💚​R💚​R🔄​f💚​R💚​R🔄​f
🔄​TestAccept/bundle/resources/clusters/deploy/simple✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
🔄​TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
🙈​TestAccept/bundle/resources/permissions🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions🟨​K🟨​K💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct🟨​K🟨​K💚​R💚​R
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform🟨​K🟨​K💚​R💚​R
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions🟨​K🟨​K💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct🟨​K🟨​K💚​R💚​R
🟨​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform🟨​K🟨​K💚​R💚​R
🔄​TestAccept/bundle/resources/synced_database_tables/basic🙈​s🙈​s✅​p✅​p🙈​s🙈​s🔄​f✅​p🙈​s🙈​s
🙈​TestAccept/bundle/run/app-with-job🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestAccept/bundle/templates/default-python/combinations/classic✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=no/PY=yes🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=yes/PY=yes🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=no🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=yes🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=yes🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.12✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.13✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13✅​p🔄​f✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p
🔄​TestGenerateFromExistingPipelineAndDeploy✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
Top 23 slowest tests (at least 2 minutes):
durationenvtestname
5:44aws-ucws linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:43azure-ucws windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:35aws windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:29gcp windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:29gcp windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:27gcp linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:13gcp linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:12aws-ucws linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:51aws-ucws windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:45aws-ucws windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:20azure-ucws windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:45azure linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
3:44azure linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:39azure windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
3:32aws windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:31azure-ucws windowsTestAccept/bundle/resources/synced_database_tables/basic
3:26azure windowsTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:17aws linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
2:53azure-ucws linuxTestAccept/bundle/resources/synced_database_tables/basic
2:25aws linuxTestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
2:15aws windowsTestSecretsPutSecretStringValue
2:14azure windowsTestSecretsPutSecretStringValue
2:11azure-ucws windowsTestSecretsPutSecretStringValue

@denik
denik enabled auto-merge November 28, 2025 13:53
@denik
denik added this pull request to the merge queueDec 1, 2025
Merged via the queue into main with commit f102a4cDec 1, 2025
13 checks passed
@denik
denik deleted the denik/task_key_order branch December 1, 2025 11:46
@denikdenik mentioned this pull request Dec 2, 2025
Merged
denik added a commit that referenced this pull request May 20, 2026
@denik
direct: treat tasks as a map identified by task_key (#4029)
3e0340f
## Changes
Configure "task_key" as a way to identify tasks in jobs.
Builds on top of #4014
## Why
Makes diffs more precise. Ignores reordering of slice without need of
sorting (#3212). Note, we still
sort, this is to be reverted in separate PR.
## Tests
Existing tests. New test that reorders tasks locally.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shreyas-goenkashreyas-goenkashreyas-goenka approved these changes
@andrewnesterandrewnesterAwaiting requested review from andrewnester
@anton-107anton-107Awaiting requested review from anton-107
@pieternpieternAwaiting requested review from pietern

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@denik@eng-dev-ecosystem-bot@shreyas-goenka