Skip to content

feat: Add OpenLineage support for non-query jobs in BigQueryInsertJobOperator - #44996

Merged
mobuchowski merged 1 commit into
apache:mainfrom
kacpermuda:feat-ol-bq-insert-job-other-job-types
Jan 3, 2025
Merged

feat: Add OpenLineage support for non-query jobs in BigQueryInsertJobOperator#44996
mobuchowski merged 1 commit into
apache:mainfrom
kacpermuda:feat-ol-bq-insert-job-other-job-types

Conversation

@kacpermuda

@kacpermudakacpermuda commented Dec 17, 2024

Copy link
Copy Markdown
Collaborator

This PR adds OpenLineage support for all BQ job types other than QUERY in BigQueryInsertJobOperator (that was already supported).


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborgboring-cyborgBot added area:providers provider:google Google (including GCP) related issues labels Dec 17, 2024
@kacpermuda
kacpermudaforce-pushed the feat-ol-bq-insert-job-other-job-types branch 4 times, most recently from 9deeff7 to 344f1ceCompareDecember 19, 2024 09:47
@kacpermuda
kacpermudaforce-pushed the feat-ol-bq-insert-job-other-job-types branch from 344f1ce to 1937e43CompareDecember 30, 2024 15:37
@kacpermuda
kacpermuda marked this pull request as ready for review December 30, 2024 16:10
@kacpermuda
kacpermudaforce-pushed the feat-ol-bq-insert-job-other-job-types branch 2 times, most recently from d7301d1 to 3fb0eb7CompareJanuary 2, 2025 10:50
@potiuk

Copy link
Copy Markdown
Member

@kacpermuda - can you please rebase that one -> we found and issue with @jscheffl with the new caching scheme - fixed in #45347 that would run "main" version of the tests. I am asking in all affected PRs to rebase.

@potiuk
potiukforce-pushed the feat-ol-bq-insert-job-other-job-types branch from 3fb0eb7 to 286fd8cCompareJanuary 2, 2025 12:22
@potiuk

Copy link
Copy Markdown
Member

Actually.. I rebased it myself :)

@kacpermuda
kacpermudaforce-pushed the feat-ol-bq-insert-job-other-job-types branch from 286fd8c to 5a0f236CompareJanuary 2, 2025 14:10
Comment threadproviders/src/airflow/providers/google/cloud/openlineage/mixins.py Outdated
Comment threadproviders/src/airflow/providers/google/cloud/openlineage/mixins.py Outdated
…Operator
Signed-off-by: Kacper Muda <mudakacper@gmail.com>
@kacpermuda
kacpermudaforce-pushed the feat-ol-bq-insert-job-other-job-types branch from 5a0f236 to 2891af3CompareJanuary 3, 2025 07:57
@mobuchowski
mobuchowski merged commit 6f9cf5a into apache:mainJan 3, 2025
@kacpermuda
kacpermuda deleted the feat-ol-bq-insert-job-other-job-types branch January 3, 2025 10:08
HariGS-DB pushed a commit to HariGS-DB/airflow that referenced this pull request Jan 16, 2025
…Operator (apache#44996)
Signed-off-by: Kacper Muda <mudakacper@gmail.com>
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
…Operator (apache#44996)
Signed-off-by: Kacper Muda <mudakacper@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providersprovider:googleGoogle (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@kacpermuda@potiuk@mobuchowski