Uh oh!
There was an error while loading. Please reload this page.
Add NodeCoordinator - #68548
Conversation
694c6ef to
3b4654eCompareuranusjr
commented
Jun 28, 2026
I think the class name should be |
Uh oh!
There was an error while loading. Please reload this page.
uranusjr
commented
Jun 28, 2026
I’m thinking, maybe the coordinator should be NodeCoordinator instead. Go does not have a GoCoordinator but a more general ExecutableCoordinator. Java has JavaCoordinator, but that’s not (mainly) because the SDK targets Java (it doesn’t just target Java but all JVM languages), but more because the command that runs the SDK bundle is called |
Uh oh!
There was an error while loading. Please reload this page.
jason810496
left a comment
There was a problem hiding this comment.
Nice! Would like to discuss the high level opening question before diving into the actual implementation, thanks.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
shivaam
commented
Jun 28, 2026
Yes, that makes sense. I will change it. |
3b4654e to
03f2d04Compare03f2d04 to
b106289CompareUh oh!
There was an error while loading. Please reload this page.
uranusjr
left a comment
There was a problem hiding this comment.
One minor request; should be good once suggestion is applied.
Uh oh!
There was an error while loading. Please reload this page.
* Add TypeScript task coordinator * Add newsfragment for TypeScript coordinator * Rename TypeScript coordinator to Node coordinator * Document Node coordinator usage for TypeScript SDK * Improve Node coordinator metadata rejection logging
This adds the Python-side NodeCoordinator needed to run TypeScript tasks through Airflow coordinator mode.
It pairs with the public TypeScript Task SDK interface in #67908. A Python Dag still defines the scheduling graph with stub tasks, while the task implementation can live in a TypeScript bundle. At execution time, Airflow routes the stub task to NodeCoordinator, which starts the Node.js bundle through the existing supervisor subprocess path.
The expected bundle layout is:
bundle.mjsis the Node.js entrypoint for the TypeScript task runtime.airflow-metadata.yamldescribes the bundle and provides the supervisor schema version the coordinator should use when talking to the runtime.Example metadata:
For this first PR, NodeCoordinator only needs
sdk.supervisor_schema_version, but this metadata file follows the same general shape as Airflow executable bundle metadata so TypeScript bundle tooling can grow into it later.Example Airflow config:
This first version keeps bundle selection deliberately simple.
bundles_rootis an ordered fallback list of directories, and the first validbundle.mjswith valid metadata is used. Dag/task-aware selection across multiple TypeScript bundles is left for follow-up work once the TypeScript bundle tooling exists.Testing
I ran a full local daemon E2E with
api-server,dag-processor, andschedulerto test this locally with my other code.Follow-up work
startCoordinator().bundle.mjsandairflow-metadata.yaml.Was generative AI tooling used to co-author this PR?
Generated-by: OpenAI Codex following the guidelines