Ensure dag.test uses serialized dag for testing - #56660

Merged
ephraimbuddy merged 5 commits into
apache:mainfrom
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged

Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom
astronomer:dont-require-serdag-for-dagtest

Conversation

@ephraimbuddy

@ephraimbuddyephraimbuddy commented Oct 15, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

While dag test command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.

This PR fixes this and ensures dag.test parses dag if the dag is not
parsed

Closes: #56657

@ephraimbuddyephraimbuddy added this to the Airflow 3.1.1 milestone Oct 15, 2025
tatiana
tatiana reviewed Oct 15, 2025
View reviewed changes

@tatianatatiana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.
Would it be worth to update the PR description to mention it closes #56657?

ashb
ashb reviewed Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py Outdated
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py Outdated
@ephraimbuddy

ephraimbuddy commented Oct 15, 2025

Copy link
Copy Markdown
ContributorAuthor

@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?

Yes. Done. Thanks

@ephraimbuddy

ephraimbuddy commented Oct 15, 2025

Copy link
Copy Markdown
ContributorAuthor

I noticed the task is failing when executing:

2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr reviewed Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py Outdated
kaxil
kaxil reviewed Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py Outdated
kaxil
kaxil reviewed Oct 16, 2025
View reviewed changes

@kaxilkaxil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?

kaxil
kaxil reviewed Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy

ephraimbuddy commented Oct 16, 2025

Copy link
Copy Markdown
ContributorAuthor

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?

dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered

@dstandish

dstandish commented Oct 16, 2025

Copy link
Copy Markdown
Contributor

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?

dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered

yeah is there much downside to just creating it?

@kaxil

kaxil commented Oct 16, 2025

Copy link
Copy Markdown
Member

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?

dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered

yeah is there much downside to just creating it?

Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase

@ephraimbuddy
ephraimbuddyforce-pushed the dont-require-serdag-for-dagtest branch from 5d8bd17 to 5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy changed the title Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy

ephraimbuddy commented Oct 17, 2025

Copy link
Copy Markdown
ContributorAuthor

I have updated this PR to enable dag parsing in the dag.test method

@ephraimbuddy
ephraimbuddyforce-pushed the dont-require-serdag-for-dagtest branch 3 times, most recently from 829c6cf to d959325CompareOctober 17, 2025 12:38
kaxil
kaxil reviewed Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py Outdated
Comment threadairflow-core/tests/unit/models/test_dag.py Outdated
Comment threadairflow-core/tests/unit/models/test_dag.py Outdated
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed the dont-require-serdag-for-dagtest branch from d959325 to 5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil approved these changes Oct 17, 2025
View reviewed changes

@kaxilkaxil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise

ephraimbuddy
ephraimbuddy commented Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py Outdated
@ephraimbuddy
ephraimbuddyforce-pushed the dont-require-serdag-for-dagtest branch 2 times, most recently from 957da39 to fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy commented Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed the dont-require-serdag-for-dagtest branch from fcc70fc to 7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy merged commit 6d977a9 into apache:mainOct 18, 2025
82 checks passed
@ephraimbuddy
ephraimbuddy deleted the dont-require-serdag-for-dagtest branch October 18, 2025 13:45
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit that referenced this pull request Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit to aaron-y-chen/airflow that referenced this pull request Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit to TyrellHaywood/airflow that referenced this pull request Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk mentioned this pull request Nov 8, 2025
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner
+1 more reviewer
@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area:CLIarea:serialization

Projects

None yet

Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Ensure dag.test uses serialized dag for testing - #56660
Merged
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Oct 18, 2025
Merged
Ensure dag.test uses serialized dag for testing#56660
ephraimbuddy merged 5 commits into
apache:mainfrom 
astronomer:dont-require-serdag-for-dagtest
Conversation
@ephraimbuddy
@ephraimbuddyephraimbuddy
commented
Oct 15, 2025•

edited

Loading

Copy link


Copy Markdown

Contributor
While dag test command uses serialized dag, dag.test was using

in-memory serialized dag making direct usage of dag.test method

resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not

parsed
Closes: #56657
@ephraimbuddyephraimbuddy
added this to the Airflow 3.1.1 milestone
Oct 15, 2025
tatiana
tatiana
reviewed
Oct 15, 2025
View reviewed changes
@tatianatatiana
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ephraimbuddy Thanks a lot for working on this so quickly.

Would it be worth to update the PR description to mention it closes #56657?
ashb
ashb
reviewed
Oct 15, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
@ephraimbuddy Thanks a lot for working on this so quickly. Would it be worth to update the PR description to mention it closes #56657?
Yes. Done. Thanks
@ephraimbuddy
ephraimbuddy
commented
Oct 15, 2025

Copy link


Copy Markdown

ContributorAuthor
I noticed the task is failing when executing:
2025-10-15T12:54:56.204021Z [info ] [DAG TEST] starting task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1336
2025-10-15T12:54:56.204589Z [info ] [DAG TEST] running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [scheduled]> [airflow.sdk.definitions.dag] loc=dag.py:1339
2025-10-15T12:54:56.210513Z [error ] [DAG TEST] Error running task <TaskInstance: my_dag.test manual__2025-10-15T12:54:56.173999+00:00 [queued]> [airflow.sdk.definitions.dag] loc=dag.py:1386
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1349, in _run_task
task_sdk_ti = TaskInstanceSDK(
File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskInstance
dag_version_id
UUID input should be a string, bytes or UUID object [type=uuid_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.11/v/uuid_type
2025-10-15T12:54:56.225062Z [info ] [DAG TEST] end task task_id=test map_index=-1 [airflow.sdk.definitions.dag] loc=dag.py:1393
2025-10-15T12:54:56.228413Z [info ] Marking run <DagRun my_dag @ 2025-10-14 00:00:00+00:00: manual__2025-10-15T12:54:56.173999+00:00, state:running, queued_at: None. run_type: manual> failed [airflow.models.dagrun.DagRun] loc=dagrun.py:1177
Dag run in failure state
Dag information:my_dag Run id: manual__2025-10-15T12:54:56.173999+00:00 Run type: manual
Failed with message: task_failure
2025-10-15T12:54:56.236897Z [info ] DagRun Finished: dag_id=my_dag, logical_date=2025-10-14 00:00:00+00:00, run_id=manual__2025-10-15T12:54:56.173999+00:00, run_start_date=2025-10-14 00:00:00+00:00, run_end_date=2025-10-15 12:54:56.228541+00:00, run_duration=132896.228541, state=failed, run_type=manual, data_interval_start=2025-10-14 00:00:00+00:00, data_interval_end=2025-10-14 00:00:00+00:00, [airflow.models.dagrun.DagRun] loc=dagrun.py:1280
DagRun failed

uranusjr
uranusjr
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/src/airflow/serialization/serialized_objects.py
Outdated

kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
kaxil
kaxil
reviewed
Oct 16, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/cli/commands/test_dag_command.py
@ephraimbuddy
ephraimbuddy
commented
Oct 16, 2025

Copy link


Copy Markdown

ContributorAuthor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
@dstandish
dstandish
commented
Oct 16, 2025

Copy link


Copy Markdown

Contributor
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.
The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
@kaxil
kaxil
commented
Oct 16, 2025

Copy link


Copy Markdown

Member
Fundamental question: Should dag.test() create a temporary serialized DAG version like it was doing in one of the early 3.0.x version, or should it bypass the serialization requirement entirely?
dag.test uses an inmemory serialized dag now which I think it was doing in the earlier version. The reason it was working before was that there was no requirement to have serialized dag in db before creating a dagrun. Now we have that requirement.

The command option( airflow dag test) creates serialized dags in DB. And I think it's ok to create it. Helps us ensure we are testing with the real value. We can do the same for dag.test if it's prefered
yeah is there much downside to just creating it?
Yeah I am in favor of making it consistent to airflow dag test . Which means change will be very isolated and not change other parts of the codebase
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
5d8bd17 to
5873cd6CompareOctober 17, 2025 08:21
@ephraimbuddyephraimbuddy
changed the title
Do not require serialized dag for dags testEnsure dag.test uses serialized dag for testingOct 17, 2025
@ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025

Copy link


Copy Markdown

ContributorAuthor
I have updated this PR to enable dag parsing in the dag.test method
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
3 times, most recently
from
829c6cf to
d959325CompareOctober 17, 2025 12:38
kaxil
kaxil
reviewed
Oct 17, 2025
View reviewed changes
Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadairflow-core/tests/unit/models/test_dag.py
Outdated

Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
d959325 to
5e9b466CompareOctober 17, 2025 12:39
kaxil
kaxil
approved these changes
Oct 17, 2025
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicks -- lgtm otherwise
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
Outdated

@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
2 times, most recently
from
957da39 to
fcc70fcCompareOctober 17, 2025 15:46
ephraimbuddy
ephraimbuddy
commented
Oct 17, 2025
View reviewed changes
Comment threadtask-sdk/src/airflow/sdk/definitions/dag.py
@ephraimbuddy
Ensure dag.test uses serialized dag for testing
bf08759
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing
c4c4907
@ephraimbuddy
Update query to v2 and do backcompat for 3.1
4ee51d4
@ephraimbuddy
Remove backcompat
670fe12
@ephraimbuddy
Return previous behaviour with SerializedDag
7f5e18c
@ephraimbuddy
ephraimbuddyforce-pushed
the
dont-require-serdag-for-dagtest
branch
from
fcc70fc to
7f5e18cCompareOctober 18, 2025 06:51
@ephraimbuddy
ephraimbuddy
merged commit 6d977a9
into
apache:mainOct 18, 2025

82 checks passed

@ephraimbuddy
ephraimbuddy
deleted the

dont-require-serdag-for-dagtest

branch
October 18, 2025 13:45
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
Ensure dag.test uses serialized dag for testing (apache#56660)
a2a24f2
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
ephraimbuddy added a commit
to astronomer/airflow
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
fixup! Ensure dag.test uses serialized dag for testing (apache#56660)
ff8f081
potiuk pushed a commit
that referenced
this pull request
Oct 18, 2025
@ephraimbuddy
[v3-1-test] Ensure dag.test uses serialized dag for testing (#56660) (#…
5202e77
…56820)
* Ensure dag.test uses serialized dag for testing (#56660)
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
(cherry picked from commit 6d977a9)
* fixup! Ensure dag.test uses serialized dag for testing (#56660)
aaron-y-chen pushed a commit
to aaron-y-chen/airflow
that referenced
this pull request
Oct 20, 2025
@ephraimbuddy@aaron-y-chen
Ensure dag.test uses serialized dag for testing (apache#56660)
5ac680f
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
TyrellHaywood pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
@ephraimbuddy@TyrellHaywood
Ensure dag.test uses serialized dag for testing (apache#56660)
58ede12
* Ensure dag.test uses serialized dag for testing
While `dag test` command uses serialized dag, dag.test was using
in-memory serialized dag making direct usage of dag.test method
resulting in error.
This PR fixes this and ensures dag.test parses dag if the dag is not
parsed
* fixup! Ensure dag.test uses serialized dag for testing
* Update query to v2 and do backcompat for 3.1
* Remove backcompat
* Return previous behaviour with SerializedDag
@potiukpotiuk
mentioned this pull request
Nov 8, 2025
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@uranusjruranusjruranusjr left review comments
@kaxilkaxilkaxil approved these changes
@bolkedebruinbolkedebruinAwaiting requested review from bolkedebruin
@amoghrajeshamoghrajeshAwaiting requested review from amoghrajeshamoghrajesh is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

+1 more reviewer

@tatianatatianatatiana left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:CLIarea:serialization

Projects


None yet


Milestone

Airflow 3.1.1

Development

Successfully merging this pull request may close these issues.

Airflow dag.test() raises AirflowException: Cannot create DagRun for DAG because the dag is not serialized

6 participants

@ephraimbuddy@dstandish@kaxil@ashb@tatiana@uranusjr