Convert `doc.yml` workflow to be reusable by webknjaz · Pull Request #103914 · python/cpython

webknjaz · 2023-04-27T05:46:09Z

This patch is meant to simplify the maintenance of multiple complex workflow definitions that are tied together into a single workflow later on.

It is separated out of #97533 for atomicity.

webknjaz

@ambv I've added a few annotations below, hopefully they're helpful

webknjaz · 2023-04-27T05:48:54Z


 concurrency:
- group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
+ group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}-reusable


This collides with the “parent” workflow if it's exactly the same, which is why it was required to introduce some difference.

webknjaz · 2023-04-27T05:49:54Z

+ Misc/**
+ .github/workflows/reusable-docs.yml
+ - name: Check for docs changes
+ if: >-


If this step is skipped, the job output above will fall back to false.

webknjaz · 2023-04-27T05:51:01Z

 runs-on: ubuntu-latest
 timeout-minutes: 10
 outputs:
+ run-docs: ${{ steps.docs-changes.outputs.run-docs || false }}


The part after || is only used as a fallback when the first part of the expression is empty (like when that step didn't even get executed).

webknjaz · 2023-04-27T05:51:29Z

+ filter: |
+ Doc/**
+ Misc/**
+ .github/workflows/reusable-docs.yml


This reimplements what was implemented through paths:.

webknjaz · 2023-04-27T05:52:28Z

+ check-docs:
+ name: 📝
+ needs: check_source
+ if: fromJSON(needs.check_source.outputs.run-docs)


The job above sets either true or false as strings, parsing that as JSON allows not having equality checks.

webknjaz · 2023-04-27T05:55:08Z

I made the workflow file name prefixed with reusable- to start a convention that would help distinguish standalone workflows from the “inclusion snippets”.

webknjaz · 2023-04-27T05:55:59Z


 on:
+ workflow_call:
 workflow_dispatch:


I kept this to make it possible to still trigger just this workflow separately, via a manual request.

@hugovk this is the only bit I wasn't sure about — is it useful to keep this trigger at all. If not, deleting it could improve maintainability..

webknjaz · 2023-04-27T05:58:40Z

- #push:
- # branches:
- # - 'main'
- # - '3.11'
- # - '3.10'
- # - '3.9'
- # - '3.8'
- # - '3.7'
- # paths:
- # - 'Doc/**'
- pull_request:
- branches:
- - 'main'
- - '3.11'
- - '3.10'
- - '3.9'
- - '3.8'
- - '3.7'
- paths:
- - 'Doc/**'
- - 'Misc/**'
- - '.github/workflows/doc.yml'


I reimplemented the filtering on the calling side. Dropping the pull_request trigger means that it won't run standalone on PRs, but it will run via inclusion into the main build workflow.

webknjaz · 2023-04-27T05:59:15Z

+name: 📝

 on:
+ workflow_call:


This bit is what allows to “include” this workflow into another.

bedevere-bot · 2023-04-30T19:28:40Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

ghost · 2023-05-02T15:45:47Z

All commit authors signed the Contributor License Agreement.

webknjaz · 2023-05-02T15:47:49Z

I have made the requested changes; please review again.

cc @ambv @hugovk

bedevere-bot · 2023-05-02T15:47:53Z

Thanks for making the requested changes!

@hugovk: please review the changes made to this pull request.

hugovk · 2023-05-10T15:33:21Z

How far back do we want to backport this (and its parent #97533), if at all?

webknjaz · 2023-05-10T15:38:21Z

@hugovk are there huge differences? I think it's okay to backport all the way back to 3.7, if the difference isn't big. Even if it is, I'm willing to explore solving all the cherry-picking conflicts, if necessary.

webknjaz · 2023-05-10T20:44:15Z

As for the "parent PR" I envision that @ambv would want it fully backported because he wanted to make auto-merge painless...

hugovk · 2023-05-11T08:20:53Z

The general policy is to only backport non-security things for bugfix branches (i.e. 3.11), but CI changes can usually go further back because we need to test older security branches (i.e. 3.7-3.10) when we do get security fixes, and keeping them in sync makes backporting easier.

https://devguide.python.org/versions/

So we're probably fine backporting to 3.7, and we can always check with the release managers.

webknjaz · 2023-05-11T09:12:31Z

Yep, that was my thinking too.

webknjaz · 2023-05-24T16:33:31Z

I've implemented a similar change in pypa/build recently, it seems to work well so far: pypa/build@04df94c.

Changes made

webknjaz · 2023-05-31T16:00:50Z

@hugovk this should fix the problem: #105151.

This is necessary because paths with whitespaces tend to crash said action[[1]][[2]][[3]]. Also, we don't need to use JSON as it's harder to parse while the value isn't used except for the emptiness check. The change fixes [[4]] [1]: https://github.com/Ana06/get-changed-files#get-all-changed-files-as-space-delimited [2]: python#103914 (comment) [3]: python#103914 (comment) [4]: python#103914 (comment)

miss-islington · 2023-07-22T13:42:52Z

Thanks @webknjaz for the PR, and @pradyunsg for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington · 2023-07-22T13:42:52Z

Thanks @webknjaz for the PR, and @pradyunsg for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

miss-islington · 2023-07-22T13:42:53Z

Sorry, @webknjaz and @pradyunsg, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 88d14da76f579fe014cbd7c15e42be4234135fe9 3.11

miss-islington · 2023-07-22T13:42:55Z

Sorry @webknjaz and @pradyunsg, I had trouble checking out the 3.12 backport branch.
Please retry by removing and re-adding the "needs backport to 3.12" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 88d14da76f579fe014cbd7c15e42be4234135fe9 3.12

miss-islington · 2023-07-22T13:43:11Z

Thanks @webknjaz for the PR, and @pradyunsg for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington · 2023-07-22T13:43:13Z

Sorry, @webknjaz and @pradyunsg, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 88d14da76f579fe014cbd7c15e42be4234135fe9 3.10

miss-islington · 2023-07-22T13:44:04Z

Thanks @webknjaz for the PR, and @pradyunsg for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington · 2023-07-22T13:44:05Z

Sorry @webknjaz and @pradyunsg, I had trouble checking out the 3.8 backport branch.
Please retry by removing and re-adding the "needs backport to 3.8" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 88d14da76f579fe014cbd7c15e42be4234135fe9 3.8

miss-islington · 2023-07-22T13:44:05Z

Thanks @webknjaz for the PR, and @pradyunsg for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

miss-islington · 2023-07-22T13:44:19Z

Sorry, @webknjaz and @pradyunsg, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 88d14da76f579fe014cbd7c15e42be4234135fe9 3.9

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> (cherry picked from commit 88d14da) Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

bedevere-bot · 2023-07-22T14:43:32Z

GH-107042 is a backport of this pull request to the 3.12 branch.

bedevere-bot · 2023-07-22T14:43:36Z

GH-107043 is a backport of this pull request to the 3.11 branch.

…) (#107043) Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>. (cherry picked from commit 88d14da) (cherry picked from commit eaa6702)

…) (#107042) Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> (cherry picked from commit 88d14da) (cherry picked from commit eaa6702)

webknjaz requested review from ezio-melotti and hugovk as code owners April 27, 2023 05:46

bedevere-bot added the awaiting review label Apr 27, 2023

webknjaz mentioned this pull request Apr 27, 2023
Introduce a gate/check GHA job #97533
Merged

webknjaz commented Apr 27, 2023
View reviewed changes

webknjaz force-pushed the maintenance/gha-reusable-docs-workflow branch 3 times, most recently from cbf7985 to 3f8d6d2Compare April 27, 2023 22:23

AlexWaygood added skip issue skip news labels Apr 28, 2023

hugovk previously requested changes Apr 30, 2023
View reviewed changes

Comment thread.github/workflows/reusable-docs.yml Outdated
Comment thread.github/workflows/build.yml Outdated

bedevere-bot removed the awaiting review label Apr 30, 2023

bedevere-bot added the awaiting changes label Apr 30, 2023

webknjaz requested a review from hugovk May 2, 2023 15:47

bedevere-bot added awaiting change review and removed awaiting changes labels May 2, 2023

webknjaz force-pushed the maintenance/gha-reusable-docs-workflow branch 2 times, most recently from 1c7f95d to 0a04207Compare May 24, 2023 16:50

bedevere-bot removed the awaiting change review label May 24, 2023

pablogsal added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Jul 22, 2023

miss-islington assigned pradyunsg Jul 22, 2023

pablogsal added the needs backport to 3.10 only security fixes label Jul 22, 2023

pablogsal added needs backport to 3.8 labels Jul 22, 2023

pablogsal removed needs backport to 3.8 needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Jul 22, 2023

webknjaz mentioned this pull request Apr 24, 2025
Please support something like "allow-failure" for a given job actions/runner#2347
Open

Uh oh!

Uh oh!

Conversation

webknjaz commented Apr 27, 2023

Uh oh!

webknjaz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Apr 30, 2023

Uh oh!

ghost commented May 2, 2023• edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webknjaz commented May 2, 2023

Uh oh!

bedevere-bot commented May 2, 2023

Uh oh!

hugovk commented May 10, 2023

Uh oh!

webknjaz commented May 10, 2023

Uh oh!

webknjaz commented May 10, 2023

Uh oh!

hugovk commented May 11, 2023

Uh oh!

webknjaz commented May 11, 2023

Uh oh!

webknjaz commented May 24, 2023

Uh oh!

webknjaz commented May 31, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

miss-islington commented Jul 22, 2023

Uh oh!

bedevere-bot commented Jul 22, 2023

Uh oh!

bedevere-bot commented Jul 22, 2023

Uh oh!

Reviewers

Assignees

ghost commented May 2, 2023•
edited by ghost
Loading