Uh oh!
There was an error while loading. Please reload this page.
UI: Return 400 instead of 500 from structure_data on malformed asset_expression - #67489
Conversation
jason810496
left a comment
There was a problem hiding this comment.
We should just re-raise the TypeError exception as HTTPException. It's fine to add more context to the exception, but we should correct the response code anyway.
Uh oh!
There was an error while loading. Please reload this page.
… is malformed
The /structure/structure_data endpoint calls get_upstream_assets() to walk the
serialized Dag's asset_expression. If the stored expression contains an unknown
key or asset type, get_upstream_assets() raises TypeError("Unsupported type: ...").
The exception escaped uncaught and FastAPI returned a generic
{"detail": "Internal Server Error"} body with no context about which Dag
triggered it, forcing operators to dig through server logs to identify the
broken Dag.
Wrap the call in try/except TypeError and re-raise as HTTPException(500) with a
detail message identifying the Dag id and version. Still a 500 (the underlying
data corruption is genuinely server-side, not bad client input), but now with a
controlled, debuggable response body.
Regression test mocks get_upstream_assets to raise TypeError and asserts the
response is 500 with a detail message that includes the Dag id.5581905 to
562a6f3ComparePer @jason810496 review feedback on apache#67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context".
jason810496
left a comment
There was a problem hiding this comment.
Please fix the static check failure, thanks.
Per @jason810496 review feedback on apache#67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context".
562a6f3 to
e299be4ComparePer @jason810496 review feedback on apache#67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context".
e299be4 to
2f65c86ComparePer @jason810496 review feedback on apache#67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context".
2f65c86 to
56ff1f7CompareCI still needs fixing. You can take a look at our contributing doc which explains how to run those locally. (pre-commit hooks etc...) if you want to reproduce locally. |
e84eb2e to
889aa1fComparedkranchii
commented
May 30, 2026
@pierrejeambrun pls review. |
choo121600
left a comment
There was a problem hiding this comment.
overall looks good :)
one nit
Uh oh!
There was an error while loading. Please reload this page.
pierrejeambrun
commented
Jun 1, 2026
Unrelated CI failure, merging. |
Uh oh!
There was an error while loading. Please reload this page.
Backport successfully created: v3-2-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
…ormed asset_expression (apache#67489) * UI: Return clear 500 detail from structure_data when asset_expression is malformed The /structure/structure_data endpoint calls get_upstream_assets() to walk the serialized Dag's asset_expression. If the stored expression contains an unknown key or asset type, get_upstream_assets() raises TypeError("Unsupported type: ..."). The exception escaped uncaught and FastAPI returned a generic {"detail": "Internal Server Error"} body with no context about which Dag triggered it, forcing operators to dig through server logs to identify the broken Dag. Wrap the call in try/except TypeError and re-raise as HTTPException(500) with a detail message identifying the Dag id and version. Still a 500 (the underlying data corruption is genuinely server-side, not bad client input), but now with a controlled, debuggable response body. Regression test mocks get_upstream_assets to raise TypeError and asserts the response is 500 with a detail message that includes the Dag id. * Use 400 BAD_REQUEST for malformed asset_expression per review feedback Per @jason810496 review feedback on apache#67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context". * Revert uv.lock diff --------- (cherry picked from commit eccbdb1) Co-authored-by: Deepak kumar <deepakkumar@meta.com> Co-authored-by: pierrejeambrun <pierrejbrun@gmail.com>
…ormed asset_expression (apache#67489) * UI: Return clear 500 detail from structure_data when asset_expression is malformed The /structure/structure_data endpoint calls get_upstream_assets() to walk the serialized Dag's asset_expression. If the stored expression contains an unknown key or asset type, get_upstream_assets() raises TypeError("Unsupported type: ..."). The exception escaped uncaught and FastAPI returned a generic {"detail": "Internal Server Error"} body with no context about which Dag triggered it, forcing operators to dig through server logs to identify the broken Dag. Wrap the call in try/except TypeError and re-raise as HTTPException(500) with a detail message identifying the Dag id and version. Still a 500 (the underlying data corruption is genuinely server-side, not bad client input), but now with a controlled, debuggable response body. Regression test mocks get_upstream_assets to raise TypeError and asserts the response is 500 with a detail message that includes the Dag id. * Use 400 BAD_REQUEST for malformed asset_expression per review feedback Per @jason810496 review feedback on apache#67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context". * Revert uv.lock diff --------- (cherry picked from commit eccbdb1) Co-authored-by: Deepak kumar <deepakkumar@meta.com> Co-authored-by: pierrejeambrun <pierrejbrun@gmail.com>
…ormed asset_expression (#67489) (#67849) * UI: Return clear 500 detail from structure_data when asset_expression is malformed The /structure/structure_data endpoint calls get_upstream_assets() to walk the serialized Dag's asset_expression. If the stored expression contains an unknown key or asset type, get_upstream_assets() raises TypeError("Unsupported type: ..."). The exception escaped uncaught and FastAPI returned a generic {"detail": "Internal Server Error"} body with no context about which Dag triggered it, forcing operators to dig through server logs to identify the broken Dag. Wrap the call in try/except TypeError and re-raise as HTTPException(500) with a detail message identifying the Dag id and version. Still a 500 (the underlying data corruption is genuinely server-side, not bad client input), but now with a controlled, debuggable response body. Regression test mocks get_upstream_assets to raise TypeError and asserts the response is 500 with a detail message that includes the Dag id. * Use 400 BAD_REQUEST for malformed asset_expression per review feedback Per @jason810496 review feedback on #67489: the malformed asset_expression ultimately originates from user-authored Dag code (via the Task SDK), so the appropriate response is 400 BAD_REQUEST rather than 500 INTERNAL_SERVER_ERROR. - Change status code from 500 to 400 in structure_data. - Add HTTP_400_BAD_REQUEST to create_openapi_http_exception_doc so the OpenAPI spec advertises the new error response. - Update regression test to assert 400 and rename accordingly. Detail message is unchanged per reviewer: "It's fine to add more context". * Revert uv.lock diff --------- (cherry picked from commit eccbdb1) Co-authored-by: Deepak kumar <deepakkumar@meta.com> Co-authored-by: pierrejeambrun <pierrejbrun@gmail.com>
The
/structure/structure_dataendpoint callsget_upstream_assets()to walk the serialized Dag'sasset_expression. If the stored expression contains an unknown key (anything other thanany/all) or an unknown asset type (anything other thanasset/asset-alias/asset-name-ref/asset-uri-ref),get_upstream_assets()raisesTypeError("Unsupported type: ...")atservices/ui/structure.py:69or:102.The exception escaped uncaught and FastAPI returned a generic
{"detail": "Internal Server Error"}response body with no context about which Dag triggered it. Operators had to dig through server logs to identify the broken Dag — a frustrating debugging loop for a multi-Dag deployment.This wraps the call in
try/except TypeErrorand re-raises asHTTPException(400)with a detail message identifying the Dag id and version. It remains a 400 because the underlying issue is genuinely server-side stored-data corruption (not bad client input — the request is valid). The improvement is the response body: it's now controlled, documented, and debuggable.Before
After
Tests
Adds
TestStructureDataEndpoint::test_should_return_400_on_malformed_asset_expressionwhich mocksget_upstream_assetsto raiseTypeErrorand asserts the response surfaces a 400 with a detail message that names the Dag and includes the wrapped error.Related
HTTPExceptionat FastAPI route boundaries #67221: "Translate domain-layer exceptions to HTTPException at FastAPI route boundaries." This PR conforms to that rule by replacing an uncaught domain exception with a controlledHTTPExceptionresponse.ui/dags.py.Was generative AI tooling used to co-author this PR?