Skip to content

Implement context accessor for DatasetEvent extra - #38481

Merged
uranusjr merged 5 commits into
apache:mainfrom
astronomer:dataset-event-extra-context
Mar 29, 2024
Merged

Implement context accessor for DatasetEvent extra#38481
uranusjr merged 5 commits into
apache:mainfrom
astronomer:dataset-event-extra-context

Conversation

@uranusjr

@uranusjruranusjr commented Mar 26, 2024•
edited
Loading

Copy link
Copy Markdown
Member

See #37810 for details. This implements the dataset_events context key to add extra to a given dataset event that’s to be emitted by the task.

I can use some help with how to do the UI part.

Lee-W
Lee-W reviewed Mar 26, 2024
View reviewed changes
Comment threadairflow/datasets/__init__.py Outdated
@uranusjr
uranusjrforce-pushed the dataset-event-extra-context branch from 54834d2 to 8380a93CompareMarch 26, 2024 07:36
@uranusjr
uranusjrforce-pushed the dataset-event-extra-context branch from 8380a93 to 712e37bCompareMarch 26, 2024 09:26
@uranusjr
uranusjr requested a review from jschefflMarch 26, 2024 10:15
@jscheffljscheffl mentioned this pull request Mar 27, 2024
Closed
jscheffl
jscheffl reviewed Mar 27, 2024
View reviewed changes

@jscheffljscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the approach as proposed here. Besides the comments to the review, can I kindly request that the existing dataset example is extended such that the "new function" is used there as a good example? Benefit would also be that there is an example existing how to retrieve the extra when receiving the event. At the moment it is a bit complex for somebody trying to figure this out.

Comment threadairflow/models/taskinstance.py Outdated
Comment threaddocs/apache-airflow/authoring-and-scheduling/datasets.rst Outdated
Comment threaddocs/apache-airflow/authoring-and-scheduling/datasets.rst Outdated
@uranusjruranusjr mentioned this pull request Mar 28, 2024
Merged
@uranusjr
uranusjrforce-pushed the dataset-event-extra-context branch from e8abd59 to 47da435CompareMarch 28, 2024 10:22
jscheffl
jscheffl reviewed Mar 28, 2024
View reviewed changes
Comment threadscripts/ci/pre_commit/pre_commit_template_context_key_sync.py Outdated
@uranusjr
uranusjrforce-pushed the dataset-event-extra-context branch from 47da435 to 2996db3CompareMarch 28, 2024 23:42
@uranusjr
uranusjr merged commit fce3a58 into apache:mainMar 29, 2024
@uranusjr
uranusjr deleted the dataset-event-extra-context branch March 29, 2024 02:06
@ephraimbuddyephraimbuddy added the type:new-feature Changelog: New Features label Mar 30, 2024
@uranusjruranusjr mentioned this pull request Apr 1, 2024
Closed
2 tasks
@uranusjruranusjr mentioned this pull request May 2, 2024
Merged
@vatsrahul1001vatsrahul1001 added this to the Airflow 2.10.0 milestone Jul 12, 2024
@bbovenzi

bbovenzi commented Jul 18, 2024

Copy link
Copy Markdown
Contributor

Covered in: #40871

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jscheffljscheffljscheffl left review comments
@potiukpotiukpotiuk approved these changes
@Lee-WLee-WLee-W approved these changes
@kaxilkaxilAwaiting requested review from kaxil
@XD-DENGXD-DENGAwaiting requested review from XD-DENG
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner
+1 more reviewer
@shrukulshrukulshrukul left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

kind:documentationtype:new-featureChangelog: New Features

Projects

None yet

Milestone

Airflow 2.10.0

Development

Successfully merging this pull request may close these issues.

8 participants

@uranusjr@bbovenzi@potiuk@Lee-W@shrukul@jscheffl@ephraimbuddy@vatsrahul1001