[v3-3-test] Release TI lock before asset listener callbacks (#70951) by github-actions[bot] · Pull Request #71065 · apache/airflow

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actions · 2026-08-04T06:58:34Z

Release TI lock before asset listener callbacks

Asset registration on the task-success path (ti_update_state) ran the
listener hooks synchronously inside the transaction holding a row lock
on the task_instance table. A slow listener, multiplied across a large
fan-out of asset events, could hold that lock for minutes, causing
statement timeouts.

The listener hooks are now deferred until the end of the endpoint
instead of executed inline during asset event creation. Registration
writes to the database still happen under the caller's transaction, so
durability is unchanged; this only moves the best-effort listener hooks
off the lock.

Optimize asset alias assoc insert
Fix exhausted iterator reuse bug
Test asset reg callback cases
(cherry picked from commit 79db995)

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

* Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

uranusjr · 2026-08-04T09:32:50Z

I found a small gap for this, fixing in #71076. The two will need to be backported together.

uranusjr · 2026-08-04T11:20:01Z

Backport for #71076 will be added to this PR.

…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

github-actionsBot mentioned this pull request Aug 4, 2026
Release TI lock before asset listener callbacks #70951
Merged

boring-cyborgBot added area:API Airflow's REST/HTTP API area:task-sdk labels Aug 4, 2026

vatsrahul1001 marked this pull request as ready for review August 4, 2026 06:59

vatsrahul1001 requested review from Lee-W, XD-DENG, amoghrajesh, ashb, kaxil and uranusjr as code owners August 4, 2026 06:59

vatsrahul1001 added this to the Airflow 3.3.1 milestone Aug 4, 2026

vatsrahul1001 added the type:bug-fix Changelog: Bug Fixes label Aug 4, 2026

Lee-W approved these changes Aug 4, 2026
View reviewed changes

uranusjr mentioned this pull request Aug 4, 2026
Explicitly rollback on task state update exception #71076
Merged

[v3-3-test] Explicitly rollback on task state update exception (#71076)
b5e4f91

uranusjr force-pushed the backport-79db995-v3-3-test branch from 2efe792 to b5e4f91Compare August 4, 2026 11:25

vatsrahul1001 merged commit e681907 into v3-3-testAug 4, 2026
75 checks passed

vatsrahul1001 deleted the backport-79db995-v3-3-test branch August 4, 2026 12:54

vatsrahul1001 mentioned this pull request Aug 7, 2026
Status of testing of Apache Airflow 3.3.1rc2 #71274
Closed