Uh oh!
There was an error while loading. Please reload this page.
[SPARK-54426][INFRA] Fix release-build.sh to detect REPO_ID correctly - #53136
[SPARK-54426][INFRA] Fix release-build.sh to detect REPO_ID correctly#53136dongjoon-hyun wants to merge 1 commit into
release-build.sh to detect REPO_ID correctly#53136Conversation
dongjoon-hyun
commented
Nov 19, 2025
Could you review this PR, @HyukjinKwon ? |
HyukjinKwon
commented
Nov 19, 2025
Thanks! |
dongjoon-hyun
commented
Nov 19, 2025
Thank you, @HyukjinKwon . Merged to master/4.1. |
…ctly ### What changes were proposed in this pull request? This PR aims to fix `release-build.sh` to detect `REPO_ID` correctly. ### Why are the changes needed? Previously, we use `grep -A 5` to find `description` tag. https://github.com/apache/spark/blob/f328b5ef14c9ef4e2d04ab69c0578ab461388715/dev/create-release/release-build.sh#L501-L502 However, it's insufficient as of now. According to Today's result, we need to grep 13 lines like the following. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 5 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> ``` ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> <repositoryURI>https://repository.apache.org/content/repositories/orgapachespark-1505</repositoryURI> <created>2025-11-16T20:23:35.413Z</created> <createdDate>Sun Nov 16 20:23:35 UTC 2025</createdDate> <createdTimestamp>1763324615413</createdTimestamp> <updated>2025-11-16T21:02:45.041Z</updated> <updatedDate>Sun Nov 16 21:02:45 UTC 2025</updatedDate> <updatedTimestamp>1763326965041</updatedTimestamp> <description>Apache Spark 4.1.0-preview4 (commit c125aea)</description> ``` ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Manually test. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" | awk '/<repositoryId>/ { id = $0 } /<description>/ && $0 ~ /Apache Spark '"$RELEASE_VERSION"'/ { print id }' <repositoryId>orgapachespark-1505</repositoryId> ``` After merging this, I'm going to test with `Apache Spark 4.1.0-preview4`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes#53136 from dongjoon-hyun/SPARK-54426. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit be281fb) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
HyukjinKwon
commented
Nov 19, 2025
Let me cherry-pick this to branch-4 and branch-3.5 too. It should work in the same way |
…ctly ### What changes were proposed in this pull request? This PR aims to fix `release-build.sh` to detect `REPO_ID` correctly. ### Why are the changes needed? Previously, we use `grep -A 5` to find `description` tag. https://github.com/apache/spark/blob/f328b5ef14c9ef4e2d04ab69c0578ab461388715/dev/create-release/release-build.sh#L501-L502 However, it's insufficient as of now. According to Today's result, we need to grep 13 lines like the following. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 5 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> ``` ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> <repositoryURI>https://repository.apache.org/content/repositories/orgapachespark-1505</repositoryURI> <created>2025-11-16T20:23:35.413Z</created> <createdDate>Sun Nov 16 20:23:35 UTC 2025</createdDate> <createdTimestamp>1763324615413</createdTimestamp> <updated>2025-11-16T21:02:45.041Z</updated> <updatedDate>Sun Nov 16 21:02:45 UTC 2025</updatedDate> <updatedTimestamp>1763326965041</updatedTimestamp> <description>Apache Spark 4.1.0-preview4 (commit c125aea)</description> ``` ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Manually test. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" | awk '/<repositoryId>/ { id = $0 } /<description>/ && $0 ~ /Apache Spark '"$RELEASE_VERSION"'/ { print id }' <repositoryId>orgapachespark-1505</repositoryId> ``` After merging this, I'm going to test with `Apache Spark 4.1.0-preview4`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes#53136 from dongjoon-hyun/SPARK-54426. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
…ctly This PR aims to fix `release-build.sh` to detect `REPO_ID` correctly. Previously, we use `grep -A 5` to find `description` tag. https://github.com/apache/spark/blob/f328b5ef14c9ef4e2d04ab69c0578ab461388715/dev/create-release/release-build.sh#L501-L502 However, it's insufficient as of now. According to Today's result, we need to grep 13 lines like the following. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 5 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> ``` ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> <repositoryURI>https://repository.apache.org/content/repositories/orgapachespark-1505</repositoryURI> <created>2025-11-16T20:23:35.413Z</created> <createdDate>Sun Nov 16 20:23:35 UTC 2025</createdDate> <createdTimestamp>1763324615413</createdTimestamp> <updated>2025-11-16T21:02:45.041Z</updated> <updatedDate>Sun Nov 16 21:02:45 UTC 2025</updatedDate> <updatedTimestamp>1763326965041</updatedTimestamp> <description>Apache Spark 4.1.0-preview4 (commit c125aea)</description> ``` No behavior change. Manually test. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" | awk '/<repositoryId>/ { id = $0 } /<description>/ && $0 ~ /Apache Spark '"$RELEASE_VERSION"'/ { print id }' <repositoryId>orgapachespark-1505</repositoryId> ``` After merging this, I'm going to test with `Apache Spark 4.1.0-preview4`. No. Closes#53136 from dongjoon-hyun/SPARK-54426. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit be281fb) Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
dongjoon-hyun
commented
Nov 19, 2025
Oh, thanks, @HyukjinKwon ! |
I cancelled the release job because other issues. Initially, For the actual testing, I'll do next week with Apache Spark 4.1.0 RC and finalizing it, @HyukjinKwon . |
dongjoon-hyun
commented
Nov 20, 2025
For 4.1.0-preview4, I'll proceed to finalize manually. |
HyukjinKwon
commented
Nov 20, 2025
thanks! |
…ctly ### What changes were proposed in this pull request? This PR aims to fix `release-build.sh` to detect `REPO_ID` correctly. ### Why are the changes needed? Previously, we use `grep -A 5` to find `description` tag. https://github.com/apache/spark/blob/f328b5ef14c9ef4e2d04ab69c0578ab461388715/dev/create-release/release-build.sh#L501-L502 However, it's insufficient as of now. According to Today's result, we need to grep 13 lines like the following. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 5 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> ``` ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" <repositoryId>orgapachespark-1505</repositoryId> <type>closed</type> <policy>release</policy> <userId>dongjoon</userId> <userAgent>curl/7.81.0</userAgent> <ipAddress>35.94.112.49</ipAddress> <repositoryURI>https://repository.apache.org/content/repositories/orgapachespark-1505</repositoryURI> <created>2025-11-16T20:23:35.413Z</created> <createdDate>Sun Nov 16 20:23:35 UTC 2025</createdDate> <createdTimestamp>1763324615413</createdTimestamp> <updated>2025-11-16T21:02:45.041Z</updated> <updatedDate>Sun Nov 16 21:02:45 UTC 2025</updatedDate> <updatedTimestamp>1763326965041</updatedTimestamp> <description>Apache Spark 4.1.0-preview4 (commit c125aea)</description> ``` ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Manually test. ``` $ curl --retry 10 --retry-all-errors -s -u "$ASF_USERNAME:$ASF_PASSWORD" https://repository.apache.org/service/local/staging/profile_repositories | grep -A 13 "<repositoryId>orgapachespark-" | awk '/<repositoryId>/ { id = $0 } /<description>/ && $0 ~ /Apache Spark '"$RELEASE_VERSION"'/ { print id }' <repositoryId>orgapachespark-1505</repositoryId> ``` After merging this, I'm going to test with `Apache Spark 4.1.0-preview4`. ### Was this patch authored or co-authored using generative AI tooling? No. Closesapache#53136 from dongjoon-hyun/SPARK-54426. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
What changes were proposed in this pull request?
This PR aims to fix
release-build.shto detectREPO_IDcorrectly.Why are the changes needed?
Previously, we use
grep -A 5to finddescriptiontag.spark/dev/create-release/release-build.sh
Lines 501 to 502 in f328b5e
However, it's insufficient as of now. According to Today's result, we need to grep 13 lines like the following.
Does this PR introduce any user-facing change?
No behavior change.
How was this patch tested?
Manually test.
After merging this, I'm going to test with
Apache Spark 4.1.0-preview4.Was this patch authored or co-authored using generative AI tooling?
No.