Skip to content

AIP-103: Passing AssetStateStoreAccessors through to BaseEventTrigger - #67839

Merged
amoghrajesh merged 29 commits into
apache:mainfrom
jroachgolf84:feature/expose-asset-state-accessor
Jun 22, 2026
Merged

AIP-103: Passing AssetStateStoreAccessors through to BaseEventTrigger#67839
amoghrajesh merged 29 commits into
apache:mainfrom
jroachgolf84:feature/expose-asset-state-accessor

Conversation

@jroachgolf84

@jroachgolf84jroachgolf84 commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Description

To ensure that classes inheriting from the BaseEventTrigger (typically used for Asset watching) are "Asset-aware", the work done in AIP-103 has been wired into the BaseEventTrigger class.

#65103 drafted an approach that "flipped" the definition of the Asset and AssetWatcher. However, for good reason, the approach was challenged. After conversation, the goal became to "pass" the AssetStateAccessors through to the BaseEventTrigger with some runtime magic.

With this model, defining an Asset and AssetWatcher remains the same as before.

fromairflow.sdkimportAsset, AssetWatcher, DAG, taskfromdatetimeimportdatetimefromtriggers.event_triggersimportGenericEventTriggergeneric_asset_watcher=AssetWatcher(
name="generic_asset_watcher",
trigger=GenericEventTrigger(
random_number=1,
waiter_delay=15
)
)
generic_asset=Asset(
name="generic_asset",
watchers=[generic_asset_watcher],
)
withDAG(
dag_id="aip_93_scoping",
start_date=datetime(2026, 1, 1),
schedule=[generic_asset]
) asdag:
@taskdefdownstream_task():
passdownstream_task()

The two PR's below were closed in favor of this PR:

Testing

No unit tests have been written for this logic yet. However, testing has been performed E2E locally with breeze. The trigger authored below is what was used for testing.

classGenericEventTrigger(BaseEventTrigger):
def__init__(
self,
random_number,
waiter_delay,
asset_name,
**kwargs
):
super().__init__(**kwargs)
self.random_number=random_numberself.waiter_delay=waiter_delayself.asset_name=asset_namedefserialize(self) ->tuple[str, dict[str, Any]]:
"""Serialize the Trigger, including the func, params, and waiter_delay."""return (
self.__class__.__module__+"."+self.__class__.__qualname__,
{
"random_number": self.random_number,
"waiter_delay": self.waiter_delay,
"asset_name": self.asset_name,
},
)
asyncdefrun(self) ->AsyncIterator[TriggerEvent]:
"""Logic that fires a TriggerEvent."""logging.info(f"***** asset_state_store: {self.asset_state_store}")
whileTrue:
result=random.randint(0, 5)
ifresult==self.random_number:
logging.info("yield'ing TriggerEvent")
yieldTriggerEvent({"status": "success", "result": result})
breaklogging.info(f"Sleeping for {self.waiter_delay} seconds")
awaitasyncio.sleep(self.waiter_delay)

Follow Up Work

This PR only injects the accessor, but does not yet wire the get/set logic. In order for the AssetStateStoreAccessor.get()/.set() to be used from within the BaseEventTrigger, additional changes in _handle_request() will need to be made. That will be done in a follow-up PR.

@boring-cyborgboring-cyborgBot added area:Executors-core LocalExecutor & SequentialExecutor area:Triggerer labels Jun 1, 2026
@jroachgolf84

Copy link
Copy Markdown
CollaboratorAuthor

cc: @cmarteepants

@jroachgolf84

Copy link
Copy Markdown
CollaboratorAuthor

cc: @vincbeck and @amoghrajesh, I closed the previous PR (#66595) after renaming the branch. We'll use this one instead.

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit but overall I like it

Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py Outdated
Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py
Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py Outdated
Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py Outdated
@jroachgolf84

Copy link
Copy Markdown
CollaboratorAuthor

Unless I get anymore feedback, I'm going to start to write tests for this.

@jroachgolf84
jroachgolf84 marked this pull request as ready for review June 5, 2026 20:35
@jroachgolf84jroachgolf84 changed the title AIP-103: Passing AssetStateAccessors through to BaseEventTriggerAIP-103: Passing AssetStateStoreAccessors through to BaseEventTriggerJun 12, 2026
@jroachgolf84

Copy link
Copy Markdown
CollaboratorAuthor

Going to wait for @amoghrajesh to merge #68438 to make changes.

@jroachgolf84jroachgolf84 added the ready for maintainer review Set after triaging when all criteria pass. label Jun 16, 2026
@amoghrajesh

Copy link
Copy Markdown
Contributor

Right, i am looking at this one now.

@amoghrajesh
amoghrajesh self-requested a review June 18, 2026 05:48

@amoghrajeshamoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pushing through this, LGTM.

Docs PR will follow?

Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py Outdated
Comment threadairflow-core/src/airflow/triggers/base.py Outdated
Comment threadairflow-core/tests/unit/triggers/test_base_trigger.py
@github-actions

Copy link
Copy Markdown
Contributor

uv.lock on main just moved via #68710 ("Add Ray constraints for Python 3.14 in Google provider"), commit 9c49080 and this PR currently conflicts.

Quickest fix:

git fetch upstream main && git rebase upstream/main
rm uv.lock && uv lock
git add uv.lock && git rebase --continue
git push --force-with-lease

Automated nudge — ignore if you're not ready to rebase. This comment is updated in place on future uv.lock bumps.

Comment threadairflow-core/src/airflow/jobs/triggerer_job_runner.py
Comment threadairflow-core/tests/unit/jobs/test_triggerer_job.py Outdated
@jroachgolf84
jroachgolf84 requested a review from kaxilJune 19, 2026 02:31
@vatsrahul1001vatsrahul1001 added this to the Airflow 3.3.0 milestone Jun 19, 2026
@jroachgolf84

Copy link
Copy Markdown
CollaboratorAuthor

@amoghrajesh - good to merge?

@amoghrajesh

Copy link
Copy Markdown
Contributor

Rebasing this one, to ensure CI stays green.

@amoghrajesh
amoghrajesh merged commit 2b71905 into apache:mainJun 22, 2026
144 of 145 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Executors-coreLocalExecutor & SequentialExecutorarea:Triggererready for maintainer reviewSet after triaging when all criteria pass.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants

@jroachgolf84@amoghrajesh@kaxil@vincbeck@vatsrahul1001