Uh oh!
There was an error while loading. Please reload this page.
Pass parameters to task when run with --python-params and python_wheel_wrapper is true - #1037
Merged
Conversation
… and python_wheel_wrapper is true
pietern
approved these changes
Dec 1, 2023
| if python_params: | ||
| params = json.loads(python_params) | ||
| except Exception as e: | ||
| print(e) |
Contributor
There was a problem hiding this comment.
This can mess with stdout; how about letting this bubble up?
ContributorAuthor
There was a problem hiding this comment.
@pietern the reason I'm not re-raising this is that dbutils.widgets.get("__python_params") throws an exception when no python params passed, so we just log it. Since we're buffering Python wheel output below, it should not impact the stdout there.
Contributor
There was a problem hiding this comment.
What kind of exception is thrown if the parameter doesn't exist? Could we make the exception we catch narrower?
ContributorAuthor
There was a problem hiding this comment.
@pietern it's Py4JJavaError, not sure if it's specific enough to be helpful in this case
Uh oh!
There was an error while loading. Please reload this page.
--python-params and python_wheel_wrapper is true--python-params and python_wheel_wrapper is true--python-params and python_wheel_wrapper is trueandrewnester
enabled auto-merge
December 1, 2023 10:30
andrewnester added a commit
that referenced
this pull request
Dec 6, 2023
CLI: * Add documentation for positional args in commands generated from the Databricks OpenAPI specification ([#1033](#1033)). * Ask for host when .databrickscfg doesn't exist ([#1041](#1041)). * Add list of supported values for flags that represent an enum field ([#1036](#1036)). Bundles: * Fix panic when bundle auth resolution fails ([#1002](#1002)). * Add versioning for bundle templates ([#972](#972)). * Add support for conditional prompting in bundle init ([#971](#971)). * Pass parameters to task when run with `--python-params` and `python_wheel_wrapper` is true ([#1037](#1037)). * Change default_python template to auto-update version on each wheel build ([#1034](#1034)). Internal: * Rewrite the friendly log handler ([#1038](#1038)). * Move bundle schema update to an internal module ([#1012](#1012)). Dependency updates: * Bump github.com/databricks/databricks-sdk-go from 0.26.0 to 0.26.1 ([#1040](#1040)).
Merged
github-merge-queueBot
pushed a commit
that referenced
this pull request
Dec 6, 2023
CLI: * Add documentation for positional args in commands generated from the Databricks OpenAPI specification ([#1033](#1033)). * Ask for host when .databrickscfg doesn't exist ([#1041](#1041)). * Add list of supported values for flags that represent an enum field ([#1036](#1036)). Bundles: * Fix panic when bundle auth resolution fails ([#1002](#1002)). * Add versioning for bundle templates ([#972](#972)). * Add support for conditional prompting in bundle init ([#971](#971)). * Pass parameters to task when run with `--python-params` and `python_wheel_wrapper` is true ([#1037](#1037)). * Change default_python template to auto-update version on each wheel build ([#1034](#1034)). Internal: * Rewrite the friendly log handler ([#1038](#1038)). * Move bundle schema update to an internal module ([#1012](#1012)). Dependency updates: * Bump github.com/databricks/databricks-sdk-go from 0.26.0 to 0.26.1 ([#1040](#1040)).
denik pushed a commit
that referenced
this pull request
May 20, 2026
…heel_wrapper` is true (#1037) ## Changes It makes the behaviour consistent with or without `python_wheel_wrapper` on when job is run with `--python-params` flag. In `python_wheel_wrapper` mode it converts dynamic `python_params` in a dynamic specially named `notebook_param` and the wrapper reads them with `dbutils` and pass to `sys.argv` Fixes#1000 ## Tests Added an integration test. Integration tests pass.
denik pushed a commit
that referenced
this pull request
May 20, 2026
CLI: * Add documentation for positional args in commands generated from the Databricks OpenAPI specification ([#1033](#1033)). * Ask for host when .databrickscfg doesn't exist ([#1041](#1041)). * Add list of supported values for flags that represent an enum field ([#1036](#1036)). Bundles: * Fix panic when bundle auth resolution fails ([#1002](#1002)). * Add versioning for bundle templates ([#972](#972)). * Add support for conditional prompting in bundle init ([#971](#971)). * Pass parameters to task when run with `--python-params` and `python_wheel_wrapper` is true ([#1037](#1037)). * Change default_python template to auto-update version on each wheel build ([#1034](#1034)). Internal: * Rewrite the friendly log handler ([#1038](#1038)). * Move bundle schema update to an internal module ([#1012](#1012)). Dependency updates: * Bump github.com/databricks/databricks-sdk-go from 0.26.0 to 0.26.1 ([#1040](#1040)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
It makes the behaviour consistent with or without
python_wheel_wrapperon when job is run with--python-paramsflag.In
python_wheel_wrappermode it converts dynamicpython_paramsin a dynamic specially namednotebook_paramand the wrapper reads them withdbutilsand pass tosys.argvFixes#1000
Tests
Added an integration test.
Integration tests pass.