Skip to content

[v3-3-test] Pin lang-SDK k8s test's Go/Java SDKs to upstream main (#69545) - #69895

Merged
jason810496 merged 1 commit into
v3-3-testfrom
backport-c22cc6d-v3-3-test
Aug 1, 2026
Merged

[v3-3-test] Pin lang-SDK k8s test's Go/Java SDKs to upstream main (#69545)#69895
jason810496 merged 1 commit into
v3-3-testfrom
backport-c22cc6d-v3-3-test

Conversation

@github-actions

Copy link
Copy Markdown
Contributor
  • Pin lang-SDK k8s test's Go/Java SDKs to upstream main

Release/backport branches often lack go-sdk/java-sdk entirely or carry a
stale, branch-cut-frozen copy, so backporting a core/task-sdk fix to such
a branch would otherwise run the lang-SDK coordinator test against a
missing or outdated SDK snapshot. The Go bundle and Java jar builds now
always fetch go-sdk/java-sdk from upstream main regardless of the checked
out branch, while airflow-core/task-sdk and the test's own harness
fixtures keep tracking whatever branch is checked out.

Fix lang-SDK Java build missing task-sdk sibling in native mode

java-sdk's sdk/build.gradle.kts reads a sibling ../task-sdk/.../schema.json
at build time. The upstream-main extraction only contains go-sdk/java-sdk,
so building from it directly (native/CI toolchain mode) failed looking for
a task-sdk directory that doesn't exist. Symlink the real, local task-sdk
alongside the extraction so the reference resolves, without pulling
task-sdk itself from upstream.

Restore lang-SDK k8s test's gradle wrapper jar dropped by export-ignore

java-sdk/.gitattributes marks gradle/wrapper/gradle-wrapper.jar export-ignore
so ASF source-release tarballs stay free of compiled binaries (LEGAL-570).
git archive, used to pull go-sdk/java-sdk out of upstream main for this test,
respects that attribute and silently drops the jar, leaving the extracted
java-sdk's ./gradlew unable to find itself and failing every build.

Shorten gradle-wrapper.jar restoration comments

  • Restore gradle wrapper scripts dropped by export-ignore in lang-SDK k8s test

Upstream main's java-sdk/.gitattributes export-ignores gradlew and
gradlew.bat (ASF LEGAL-570) in addition to the wrapper jar, so the
git-archive extraction of upstream sources no longer contains the
wrapper script the Java build invokes, failing CI with
FileNotFoundError: './gradlew'. Restore all three wrapper files from
the pinned commit instead of only the jar.

  • Fix breeze k8s setup-lang-sdk-test crash with --dry-run

In dry-run mode run_command skips execution and returns stdout="" (a
str), so every filesystem step consuming a command's output crashed:
write_bytes on the gradle wrapper restore raised TypeError, the Go
bundle copytree/copy raised FileNotFoundError, the Java jar glob hit
sys.exit(1), and the artifact upload raised StopIteration. Guard those
steps so --dry-run prints the full provisioning command sequence and
completes cleanly.
(cherry picked from commit c22cc6d)

Co-authored-by: Jason(Zhe-You) Liu 68415893+jason810496@users.noreply.github.com

@vatsrahul1001

Copy link
Copy Markdown
Contributor

@jason810496 docs check failing

…9545)
* Pin lang-SDK k8s test's Go/Java SDKs to upstream main
Release/backport branches often lack go-sdk/java-sdk entirely or carry a
stale, branch-cut-frozen copy, so backporting a core/task-sdk fix to such
a branch would otherwise run the lang-SDK coordinator test against a
missing or outdated SDK snapshot. The Go bundle and Java jar builds now
always fetch go-sdk/java-sdk from upstream main regardless of the checked
out branch, while airflow-core/task-sdk and the test's own harness
fixtures keep tracking whatever branch is checked out.
Fix lang-SDK Java build missing task-sdk sibling in native mode
java-sdk's sdk/build.gradle.kts reads a sibling ../task-sdk/.../schema.json
at build time. The upstream-main extraction only contains go-sdk/java-sdk,
so building from it directly (native/CI toolchain mode) failed looking for
a task-sdk directory that doesn't exist. Symlink the real, local task-sdk
alongside the extraction so the reference resolves, without pulling
task-sdk itself from upstream.
Restore lang-SDK k8s test's gradle wrapper jar dropped by export-ignore
java-sdk/.gitattributes marks gradle/wrapper/gradle-wrapper.jar export-ignore
so ASF source-release tarballs stay free of compiled binaries (LEGAL-570).
git archive, used to pull go-sdk/java-sdk out of upstream main for this test,
respects that attribute and silently drops the jar, leaving the extracted
java-sdk's ./gradlew unable to find itself and failing every build.
Shorten gradle-wrapper.jar restoration comments
* Restore gradle wrapper scripts dropped by export-ignore in lang-SDK k8s test
Upstream main's java-sdk/.gitattributes export-ignores gradlew and
gradlew.bat (ASF LEGAL-570) in addition to the wrapper jar, so the
git-archive extraction of upstream sources no longer contains the
wrapper script the Java build invokes, failing CI with
FileNotFoundError: './gradlew'. Restore all three wrapper files from
the pinned commit instead of only the jar.
* Fix breeze k8s setup-lang-sdk-test crash with --dry-run
In dry-run mode run_command skips execution and returns stdout="" (a
str), so every filesystem step consuming a command's output crashed:
write_bytes on the gradle wrapper restore raised TypeError, the Go
bundle copytree/copy raised FileNotFoundError, the Java jar glob hit
sys.exit(1), and the artifact upload raised StopIteration. Guard those
steps so --dry-run prints the full provisioning command sequence and
completes cleanly.
(cherry picked from commit c22cc6d)
Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
@potiuk
potiukforce-pushed the backport-c22cc6d-v3-3-test branch from c8156ec to 25decc0CompareJuly 30, 2026 19:37
@jason810496

Copy link
Copy Markdown
Member

@jason810496 docs check failing

IIUC, other PR fixed it, I will merge this one.
Then backports of #70561 and #70824 should fix the Lang-SDK k8s test on v3-3-test.

@jason810496
jason810496 merged commit ac9e8cf into v3-3-testAug 1, 2026
112 of 114 checks passed
@jason810496
jason810496 deleted the backport-c22cc6d-v3-3-test branch August 1, 2026 14:14
@vatsrahul1001vatsrahul1001 added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Aug 3, 2026
@vatsrahul1001vatsrahul1001 added this to the Airflow 3.3.1 milestone Aug 3, 2026
vatsrahul1001 pushed a commit that referenced this pull request Aug 5, 2026
…9545) (#69895)
* Pin lang-SDK k8s test's Go/Java SDKs to upstream main
Release/backport branches often lack go-sdk/java-sdk entirely or carry a
stale, branch-cut-frozen copy, so backporting a core/task-sdk fix to such
a branch would otherwise run the lang-SDK coordinator test against a
missing or outdated SDK snapshot. The Go bundle and Java jar builds now
always fetch go-sdk/java-sdk from upstream main regardless of the checked
out branch, while airflow-core/task-sdk and the test's own harness
fixtures keep tracking whatever branch is checked out.
Fix lang-SDK Java build missing task-sdk sibling in native mode
java-sdk's sdk/build.gradle.kts reads a sibling ../task-sdk/.../schema.json
at build time. The upstream-main extraction only contains go-sdk/java-sdk,
so building from it directly (native/CI toolchain mode) failed looking for
a task-sdk directory that doesn't exist. Symlink the real, local task-sdk
alongside the extraction so the reference resolves, without pulling
task-sdk itself from upstream.
Restore lang-SDK k8s test's gradle wrapper jar dropped by export-ignore
java-sdk/.gitattributes marks gradle/wrapper/gradle-wrapper.jar export-ignore
so ASF source-release tarballs stay free of compiled binaries (LEGAL-570).
git archive, used to pull go-sdk/java-sdk out of upstream main for this test,
respects that attribute and silently drops the jar, leaving the extracted
java-sdk's ./gradlew unable to find itself and failing every build.
Shorten gradle-wrapper.jar restoration comments
* Restore gradle wrapper scripts dropped by export-ignore in lang-SDK k8s test
Upstream main's java-sdk/.gitattributes export-ignores gradlew and
gradlew.bat (ASF LEGAL-570) in addition to the wrapper jar, so the
git-archive extraction of upstream sources no longer contains the
wrapper script the Java build invokes, failing CI with
FileNotFoundError: './gradlew'. Restore all three wrapper files from
the pinned commit instead of only the jar.
* Fix breeze k8s setup-lang-sdk-test crash with --dry-run
In dry-run mode run_command skips execution and returns stdout="" (a
str), so every filesystem step consuming a command's output crashed:
write_bytes on the gradle wrapper restore raised TypeError, the Go
bundle copytree/copy raised FileNotFoundError, the Java jar glob hit
sys.exit(1), and the artifact upload raised StopIteration. Guard those
steps so --dry-run prints the full provisioning command sequence and
completes cleanly.
(cherry picked from commit c22cc6d)
Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-toolsarea:kubernetes-testschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@vatsrahul1001@jason810496