Warn on suspicious Dag and task IDs at Java SDK build time

[Andrushika]

Following the review here, hard validation in the SDK looks like the wrong fit. The allowed ID character set has changed before, and the Airflow server already validates it authoritatively. So instead of failing at registration, this warns at build time from the annotation processor when a @Builder.Dag or @Builder.Task ID would be rejected by the server (longer than 250 chars, or characters outside letters, digits, -, ., _). The build still succeeds and the message points at the server as the source of truth, so it is best-effort only. Same approach as the Go SDK in #69965.

related: #69400, #69965

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Code (Opus 4.8) following the guidelines

[jason810496]

Thank for the fix! I found that I need to fix the coordinator layer first (the interface to spawn the lang-sdk side subprocess) so both #69965 and this one need to be on hold for a moment.

I will share more context here soon and let you know once the coordinator side patch merged. Thanks.

[uranusjr]

If you use an invalid dag id, it would match nothing on the Python (stub) side. We can perhaps add some detection logic if you have a potential dag (or task) id mismatch between the stub and implementation, but honestly I’m not convinced we need to actively check for the character set. Doing this also adds the potential that the check may become stale if the allowed character set changes (it changed before).

If we really want to checks this, I would want to do this in the dag processor, and only in Python (i.e. the JAR or executable needs to send the id to Python to be validated, instead of validating directly in the language sdk).

[jason810496]

If you use an invalid dag id, it would match nothing on the Python (stub) side. We can perhaps add some detection logic if you have a potential dag (or task) id mismatch between the stub and implementation, but honestly I’m not convinced we need to actively check for the character set. Doing this also adds the potential that the check may become stale if the allowed character set changes (it changed before).

I see. It makes sense not to validate for the case of Mixed Lang Dag. From my perspective, the validation should be useful for the upcoming native Dag so no harm to introduce it. (Or we can on hold until the native Dag out) WDYT? Thanks.

[Andrushika]

Oh, I see...... I have no problem putting this on hold or even closing it.

The reason I opened this PR is that I thought the behavior across lang SDKs should be aligned. I saw #69400 in ts-sdk, so I thought it would be good to introduce the same validation in java-sdk.

If we really want to checks this, I would want to do this in the dag processor, and only in Python

For the current mixed-lang Dag, I agree that the Python stub is the source of truth, so validation in each lang SDK may not be needed.

the validation should be useful for the upcoming native Dag so no harm to introduce it.

If we need validation in lang SDKs for native Dag in the future, the discrepancy TP mentioned would still exist. In that case, maybe we can keep the valid and invalid Dag/task IDs as shared test cases and let all the lang SDKs use them, instead of defining the expected behavior separately in each SDK?

[potiuk]

@Andrushika — There are 1 unresolved review thread(s) on this PR from @jason810496. Could you either push a fix or reply in each thread explaining why the feedback doesn't apply? Once you believe the feedback is addressed, mark the thread as resolved so the reviewer isn't re-pinged needlessly. Thanks!

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[uranusjr]

the validation should be useful for the upcoming native Dag so no harm to introduce it

Even in this case, the validation should happen in either the dag processor or the dag submission API endpoint (and surfaced in the dag processor), for the same reasons I mentioned in the second paragraph. This should not be implemented in every language.

[uranusjr]

I see some of the PRs have already been merged without this discussion being resolved. They should be reverted.

[Andrushika]

Hi TP! Here's the related PR, I tried to solve the same problem in go-sdk, and Jason gave some suggestion here: #69965 (review)

Jason suggests the "compile-time" best-effort validation: only shows a warning when an invalid id detected. I think this could prevent drift and preserve the source of truth while giving appropriate hints when users compile their lang-sdk stub.
I'm not sure if it meets expectations. If not, I can close this PR.

I've implemented in java-sdk and force-pushed in the latest commit currently:
https://github.com/apache/airflow/pull/69937/changes#diff-8192c67224a3e477f432eaecc29f4978a37b653fc176d9c6b5f38d17111b4971R205-R232

Thanks!

[Andrushika]

Hi @uranusjr@jason810496, just a gentle ping. Since the same validation has already been added to the Go SDK (#69965) and ts SDK (#70993), do you think the Java SDK should follow the same behavior?

I’ve implemented it in the latest commit. Could you please take a look when you have time? Thanks!

[jason810496]

Thanks for the update!

We need to validate the interface based Dag syntax as well. Actually, we only need to validate at the interface based syntax as the annotation based syntax will be transformed into interface based syntax anyway. Thanks.

[jason810496]

It would be bette to move the validation into a dedicated module since we need to validation the interface based Dag as well.

[Andrushika]

Thanks! I’ve pushed a commit that does the check in Bundle’s init, so both the interface based and annotation based syntaxes are covered. But this means the warnings show up at runtime in the task logs, not at build time. It is different from what we discussed above, and also different from the implementation in ts-sdk and go-sdk.

This is because the java-sdk bundle does not have the ids in its manifest, so doing the validation at build time requires executing the built bundle to extract them.

If the build-time validation is strictly needed, I guess we need a similar introspection mode that runs the built bundle and returns the IDs to the Gradle plugin, as ts-sdk and go-sdk do. WDYT?

[jason810496]

Good point. I think the Gradle-native shape for this would be a dedicated verification task.

Could we add something like inspectAirflowBundle / checkAirflowBundle that runs after classes, collects the Dag/task IDs from the assembled Bundle, and wire it into Gradle’s check lifecycle?

That way users can choose the level they want:

• ./gradlew checkAirflowBundle to run only the Airflow bundle check
• ./gradlew check to include it with normal verification
• ./gradlew check -x checkAirflowBundle or a project property to skip it

This also gives us a better place to print more verbose guidance. Compilation can stay focused on source generation/compile errors, while the Airflow-specific validation lives in an explicit verification task.

The important bit is that the task probably still needs a BundleInspector-style path that instantiates the BundleBuilder and reads the resulting Bundle. Somehow like:

airflow/java-sdk/adr/0003-pure-java-dags.md

Lines 57 to 100 in 8fd35a1

	### Build-Time Metadata: `airflow-metadata.yaml`
	At build time, the SDK runs `BundleInspector` — a build-time utility that reflectively instantiates the user's `BundleBuilder` class, calls `getDags()`, and writes a YAML file listing every DAG ID and its task IDs:
	```yaml
	dags:
	java_example:
	tasks:
	- extract
	- transform
	- load
	```
	This file is embedded in the JAR root and referenced by the `Airflow-Java-SDK-Metadata` manifest attribute.
	**Why build-time, not runtime?** The metadata must be available before the JVM starts. `BundleScanner` reads it from the JAR to discover which DAG IDs a bundle contains — this is used for `@task.stub` routing (mapping a `dag_id` to the correct bundle's classpath) without paying JVM startup cost. For pure Java DAGs, the coordinator already knows the bundle path, but the metadata is still useful for validation and tooling.
	**`BundleInspector`:**
	```kotlin
	object BundleInspector {
	@JvmStatic
	fun main(args: Array<String>) {
	val className = args[0]
	val outputPath = args[1]
	val clazz = Class.forName(className)
	val instance = clazz.getDeclaredConstructor().newInstance() as? BundleBuilder
	?: error("$className does not implement BundleBuilder")
	val dags = instance.getDags()
	File(outputPath).apply { parentFile.mkdirs() }.writeText(toYaml(dags))
	}
	internal fun toYaml(dags: List<Dag>): String = buildString {
	appendLine("dags:")
	for (dag in dags) {
	appendLine(" ${dag.dagId}:")
	appendLine(" tasks:")
	for (taskId in dag.tasks.keys) {
	appendLine(" - $taskId")
	}
	}
	}
	}
	```

Thanks.