Skip to content

AIP-103: Add Core API endpoints for task state and asset state - #67041

Merged
amoghrajesh merged 7 commits into
apache:mainfrom
astronomer:aip-103-6-core-api
May 20, 2026
Merged

AIP-103: Add Core API endpoints for task state and asset state#67041
amoghrajesh merged 7 commits into
apache:mainfrom
astronomer:aip-103-6-core-api

Conversation

@amoghrajesh

Copy link
Copy Markdown
Contributor

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

closes:

Proposed change

Adds public REST endpoints for task state and asset state under /api/v2.

Task State — scoped to a task instance within a DAG run:

GET /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/state
GET /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/state/{key}
PUT /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/state/{key}
DELETE /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/state/{key}
DELETE /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/state

map_index query param (default -1). ?all_map_indices=true on the clear endpoint wipes state across all mapped instances of the task.

Asset State — cross-run, scoped to an asset:

GET /api/v2/assets/{asset_id}/state
GET /api/v2/assets/{asset_id}/state/{key}
PUT /api/v2/assets/{asset_id}/state/{key}
DELETE /api/v2/assets/{asset_id}/state/{key}
DELETE /api/v2/assets/{asset_id}/state

URL patterns follow existing conventions — task state mirrors XCom, asset state mirrors /assets/{asset_id}/.

Testing

Task State APIs

  1. set API
imageimage
  1. List API
image
  1. get API
image
  1. delete API
imageimage
  1. clear API

For this first add couple of task states for mapped task

image

Without all_map_indices flag (only current deleted)

image

With the all_map_indices flag:

imageimage

Asset State APIs

Using id=4 to test

image
  1. set API
imageimage
  1. get API
image
  1. list API
image
  1. delete API
imageimage
  1. To test clear, will add a few asset states
imageimage

All clear!

image
  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Comment threadairflow-core/src/airflow/api_fastapi/core_api/datamodels/task_state.py Outdated

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! LGTM overall, thanks. There're same questions for the task state endpoint.

Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py Outdated

@pierrejeambrunpierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nits. Looking good overall

Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py Outdated
Comment threadairflow-core/src/airflow/api_fastapi/core_api/datamodels/task_state.py Outdated
@amoghrajeshamoghrajesh moved this from Backlog to Ready in AIP-103: Task State ManagementMay 20, 2026
@amoghrajeshamoghrajesh moved this from Ready to In review in AIP-103: Task State ManagementMay 20, 2026
@amoghrajesh

Copy link
Copy Markdown
ContributorAuthor

The failing tests in https://github.com/apache/airflow/actions/runs/26144521163/job/76898272728?pr=67041, seem to be unrelated to these changes.

@amoghrajesh

Copy link
Copy Markdown
ContributorAuthor

Merging this one in, thanks for reviews!

@amoghrajesh
amoghrajesh merged commit f9e4e6e into apache:mainMay 20, 2026
138 of 141 checks passed
@amoghrajesh
amoghrajesh deleted the aip-103-6-core-api branch May 20, 2026 10:48
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: airflow-ctl/v0-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

StatusBranchResult
airflow-ctl/v0-1-testCommit Link

You can attempt to backport this manually by running:

cherry_picker f9e4e6e airflow-ctl/v0-1-test

This should apply the commit to the airflow-ctl/v0-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@amoghrajeshamoghrajesh mentioned this pull request May 20, 2026
1 task
@kaxil

Copy link
Copy Markdown
Member

The failing tests in https://github.com/apache/airflow/actions/runs/26144521163/job/76898272728?pr=67041, seem to be unrelated to these changes.

Do you think it was because of PR behind main. Or trying to think why langchain wasn't installed

@amoghrajesh

Copy link
Copy Markdown
ContributorAuthor

@kaxil it was an actual issue fixed via #67237

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:airflow-ctlarea:APIAirflow's REST/HTTP APIarea:UIRelated to UI/UX. For Frontend Developers.backport-to-airflow-ctl/v0-1-test

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants

@amoghrajesh@kaxil@pierrejeambrun@jason810496@jscheffl