Skip to content

Fix for pip flags with equal sign being incorrectly treated as local file names - #3766

Merged
andrewnester merged 2 commits into
mainfrom
fix/env-spec-double-dash
Oct 15, 2025
Merged

Fix for pip flags with equal sign being incorrectly treated as local file names#3766
andrewnester merged 2 commits into
mainfrom
fix/env-spec-double-dash

Conversation

@andrewnester

Copy link
Copy Markdown
Contributor

Changes

Fix for pip flags with equal sign being incorrectly treated as local file names

Why

Pip flags containing an equal sign and used in the environment dependencies section were incorrectly treated as local file names, leading to the bundle deploy error.

Tests

Added test case for this

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 15, 2025

Copy link
Copy Markdown
Collaborator

Run: 18526081168

Env🔄​flaky💚​RECOVERED✅​pass🙈​skip
💚​aws linux1321545
💚​aws windows1322544
💚​aws-ucws linux1437441
💚​aws-ucws windows1438440
💚​azure linux1321544
💚​azure windows1322543
💚​azure-ucws linux1437440
💚​azure-ucws windows1438439
🔄​gcp linux31317546
🔄​gcp windows31318545
7 failing tests:
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
TestAccept/bundle/deployment/bind/cluster✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/deployment/bind/dashboard✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/resources/pipelines/update✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/resources/pipelines/update/DATABRICKS_BUNDLE_ENGINE=direct-exp✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/templates/default-python/integration_classic✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p

// Alternatively it could be a flag with no argument and no space after it
// For example: -r myfile.txt or --index-url http://myindexurl.com or -i
re := regexp.MustCompile(`(^|\s+)--?[a-zA-Z0-9-]+(\s+|$)`)
re := regexp.MustCompile(`(^|\s+)--?[a-zA-Z0-9-]+(([\s|=]+)|$)`)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think you only need ([\s=]|$) -- either a space, an =, or the end of the string.

More than one space or = doesn't change the result.

@andrewnester
andrewnester added this pull request to the merge queueOct 15, 2025
Merged via the queue into main with commit 20dfac0Oct 15, 2025
13 checks passed
@andrewnester
andrewnester deleted the fix/env-spec-double-dash branch October 15, 2025 11:30
deco-sdk-taggingBot added a commit that referenced this pull request Oct 16, 2025
## Release v0.273.0
### Notable Changes
* (via Terraform v1.92.0) DABs will no longer try to update pipeline permissions upon pipeline deletion. This fixes PERMISSION\_ERROR upon 'bundle destroy'
for pipelines that have run\_as setting enabled (described in https://community.databricks.com/t95/data-engineering/dab-dlt-destroy-fails-due-to-ownership-permissions-mismatch/td-p/132101)
The downside is that if 'permissions:' block is removed from the resource, DABs will not try anymore to restore permissions to just the owner of the pipeline.
### CLI
* Add the `--configure-serverless` flag to `databricks auth login` to configure Databricks Connect to use serverless.
### Dependency updates
* Upgrade Go SDK to 0.82.0 ([#3769](#3769))
* Upgrade TF provider to 1.92.0 ([#3772](#3772))
### Bundles
* Updated the internal lakeflow-pipelines template to use an "src" layout ([#3671](#3671)).
* Fix for pip flags with equal sign being incorrectly treated as local file names ([#3766](#3766))
denik pushed a commit that referenced this pull request May 20, 2026
…file names (#3766)
## Changes
Fix for pip flags with equal sign being incorrectly treated as local
file names
## Why
Pip flags containing an equal sign and used in the environment
dependencies section were incorrectly treated as local file names,
leading to the bundle deploy error.
## Tests
Added test case for this
<!-- If your PR needs to be included in the release notes for next
release,
add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
denik pushed a commit that referenced this pull request May 20, 2026
## Release v0.273.0
### Notable Changes
* (via Terraform v1.92.0) DABs will no longer try to update pipeline permissions upon pipeline deletion. This fixes PERMISSION\_ERROR upon 'bundle destroy'
for pipelines that have run\_as setting enabled (described in https://community.databricks.com/t95/data-engineering/dab-dlt-destroy-fails-due-to-ownership-permissions-mismatch/td-p/132101)
The downside is that if 'permissions:' block is removed from the resource, DABs will not try anymore to restore permissions to just the owner of the pipeline.
### CLI
* Add the `--configure-serverless` flag to `databricks auth login` to configure Databricks Connect to use serverless.
### Dependency updates
* Upgrade Go SDK to 0.82.0 ([#3769](#3769))
* Upgrade TF provider to 1.92.0 ([#3772](#3772))
### Bundles
* Updated the internal lakeflow-pipelines template to use an "src" layout ([#3671](#3671)).
* Fix for pip flags with equal sign being incorrectly treated as local file names ([#3766](#3766))
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewnester@eng-dev-ecosystem-bot@pietern