Skip to content

[Python] Support pipelines - #2574

Merged
kanterov merged 4 commits into
mainfrom
python-support-pipelines
Apr 8, 2025
Merged

[Python] Support pipelines#2574
kanterov merged 4 commits into
mainfrom
python-support-pipelines

Conversation

@kanterov

@kanterovkanterov commented Mar 26, 2025

Copy link
Copy Markdown
Collaborator

Changes

Add support for pipelines into experimental/python.

Only the name property is supported, and the generated code will be added as a follow-up.

Why

It allows using pipelines in addition to jobs.

Tests

Unit and acceptance tests

@kanterov
kanterovforce-pushed the python-support-pipelines branch from 82f8d25 to 905203fCompareMarch 31, 2025 08:57
@kanterov
kanterovforce-pushed the python-support-pipelines branch from 905203f to 095303dCompareMarch 31, 2025 08:58
@kanterov
kanterovforce-pushed the python-support-pipelines branch from 095303d to 30d961bCompareApril 2, 2025 19:06
@kanterov
kanterovforce-pushed the python-support-pipelines branch from 30d961b to ba9648aCompareApril 3, 2025 07:42
@kanterov
kanterov marked this pull request as ready for review April 3, 2025 08:56
@kanterovkanterov changed the title [Python] Support pipelines (WIP)[Python] Support pipelinesApr 3, 2025
name: VariableOrOptional[str]
"""
TODO
"""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this TODO for? can we drop this completely?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pipeline class doesn't have any fields except name, which is only used as an example. There is a separate PR to add remaining fields.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you link this to this PR (or the follow-up if ready)?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 , linked

PR with generated code: #2639

Comment threadexperimental/python/databricks/bundles/pipelines/_models/pipeline.py Outdated
Comment threadacceptance/bundle/python/pipelines-support/test.toml Outdated
@kanterov
kanterov requested a review from denikApril 4, 2025 12:38

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, bar the comments.

Comment threadacceptance/bundle/python/pipelines-support/test.toml Outdated
Comment threadacceptance/bundle/python/pipelines-support/databricks.yml Outdated
name: VariableOrOptional[str]
"""
TODO
"""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you link this to this PR (or the follow-up if ready)?

@kanterov
kanterovforce-pushed the python-support-pipelines branch from ba9648a to dfc8de5CompareApril 8, 2025 07:43
@kanterov
kanterov requested a review from pieternApril 8, 2025 07:45
@kanterov

Copy link
Copy Markdown
CollaboratorAuthor

@pietern addressed comments, please take a look again

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment threadacceptance/bundle/python/test.toml Outdated
@pietern
pieterntemporarily deployed to test-trigger-is April 8, 2025 10:49 — with GitHub Actions Inactive
@kanterov
kanterov added this pull request to the merge queueApr 8, 2025
Merged via the queue into main with commit 8b7aa37Apr 8, 2025
@kanterov
kanterov deleted the python-support-pipelines branch April 8, 2025 14:19
kanterov added a commit that referenced this pull request Apr 8, 2025
## Changes
Add support for pipelines into `experimental/python`.
Only the `name` property is supported, and the generated code will be
added as a follow-up.
## Why
It allows using pipelines in addition to jobs.
## Tests
Unit and acceptance tests
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
github-merge-queueBot pushed a commit that referenced this pull request Apr 9, 2025
## Changes
Add generated code for pipelines support introduced in
#2574
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
Add support for pipelines into `experimental/python`.
Only the `name` property is supported, and the generated code will be
added as a follow-up.
## Why
It allows using pipelines in addition to jobs.
## Tests
Unit and acceptance tests
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
Add generated code for pipelines support introduced in
#2574
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@kanterov@pietern@denik