ARROW-6917: [Archery][Release] Add support for JIRA curation, changelog generation and commit cherry-picking for maintenance releases

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.

[kszucs]

Also aims to resolve https://issues.apache.org/jira/browse/ARROW-8456

I'd like to cover it (at least partly) with tests, but now I'm using it to create the maintenance branch for 0.17.1.

It also supports (re-)generating changelogs based on the jira and commit curation of each release including PARQUET issues.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-6917

[wesm]

Do you still want to write unit tests or can we merge this and address the next time we need to make a patch release?

[kszucs]

I don’t want to have more untested code in the build scripts so yes, I’d like to have a minimal coverage there (even if it is painful to mock).

[kszucs]

Also updated the changelog generation and removed the other two implementation.

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

If there is no objection then I'd like to use it for curating jira and generating the changelogs for 1.0.

[wesm]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

[nealrichardson]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

[kszucs]

I see on #7769 you called archery release --jira-cache /tmp/archery-cache changelog regenerate. Could you please document this somewhere? It's a bit non-obvious.

I'm going to update the release mgmt guide during the release.

[kszucs]

@wesm I'm not sure how to add tests this easily, but we can certainly defer that to a follow-up.

Well, the business logic that's unrelated to accessing remote data could be unit tested (e.g. the remote querying would be mocked out), but that would require some refactoring. For example, you've seen how I wrote unit tests for the dev/merge_arrow_py.py script?

Yeah I'd need to mock both jira issues and git history including branches and tags which will take some time, so I'm deferring it to a follow-up after the release.

[kszucs]

Follow-ups:

• Testing: https://issues.apache.org/jira/browse/ARROW-9487
• Update the post release website script to use the new changelog generation (will handle it after the RCs) https://issues.apache.org/jira/browse/ARROW-9488

[kszucs]

+1, merging since I'm going to utilize this for creating 1.0 release.