Skip to content

Added experimental.skip_artifact_cleanup flag - #2980

Merged
andrewnester merged 8 commits into
mainfrom
feature/skip-artifact-cleanup
Jun 3, 2025
Merged

Added experimental.skip_artifact_cleanup flag#2980
andrewnester merged 8 commits into
mainfrom
feature/skip-artifact-cleanup

Conversation

@andrewnester

@andrewnesterandrewnester commented Jun 3, 2025

Copy link
Copy Markdown
Contributor

Changes

Added experimental.skip_artifact_cleanup flag. When set to true .internal folder in Databricks workspace which contains built artifacts such as wheels won't be cleaned up on deploy

Fixes#2969#2958

Why

It can be useful to keep old built wheels around for cases when they are running jobs using the older version of wheels to prevent new deployments from breaking these jobs.

Tests

Added acceptance tests

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The singular name, skip_artifact_cleanup, sounds more natural to me, as it skips the process of "artifact cleanup."

Comment threadbundle/artifacts/upload.go Outdated
@andrewnesterandrewnester changed the title Added experimental.skip_artifacts_cleanup flagAdded experimental.skip_artifact_cleanup flagJun 3, 2025
@andrewnester
andrewnester merged commit b60579b into mainJun 3, 2025
@andrewnester
andrewnester deleted the feature/skip-artifact-cleanup branch June 3, 2025 13:05
deco-sdk-taggingBot added a commit that referenced this pull request Jun 3, 2025
## Release v0.254.0
### Bundles
* Added `experimental.skip_artifact_cleanup` flag ([#2980](#2980))
* Add an experimental project template for Lakeflow Declarative Pipelines ([#2959](#2959))
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
Added `experimental.skip_artifact_cleanup` flag. When set to `true`
.internal folder in Databricks workspace which contains built artifacts
such as wheels won't be cleaned up on deploy
Fixes#2969#2958
## Why
It can be useful to keep old built wheels around for cases when they are
running jobs using the older version of wheels to prevent new
deployments from breaking these jobs.
## Tests
Added acceptance tests
<!-- If your PR needs to be included in the release notes for next
release,
add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
denik pushed a commit that referenced this pull request May 20, 2026
## Release v0.254.0
### Bundles
* Added `experimental.skip_artifact_cleanup` flag ([#2980](#2980))
* Add an experimental project template for Lakeflow Declarative Pipelines ([#2959](#2959))
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DABs cleanup of dist and .internal causing locked wheel versions to fail

2 participants

@andrewnester@pietern