Required checks will stop to be required if their name later on become shadowed by a faster job in another workflow

[Daskeladden]

Describe the bug
Required checks will stop to be required if their name later on become shadowed by a faster job in another workflow

To Reproduce
Steps to reproduce the behavior:

1. Make a workflow "A" with a job named "Test" and make that job take 1 minute and then always fails
2. Go into GitHub actions branch protection rule and make "Test" required for merge to master
3. Make a branch from master and push an empty commit
4. Make a PR
5. See that the PR is blocked from being merged
6. In the branch, add a new workflow "B" and a job called "Test" and make it succeed instantly.
7. Push empty commit to branch
8. Observe that the Merge button becomes active and clickable once the job "Test" from workflow B has completed even though the job "Test" from workflow "A" is still running.

Expected behavior
PR should be blocked from being merged until all jobs named "Test" have completed successfully.

Runner Version and Platform

Runner version: 2.278.0
Self hosted runner Linux Ubuntu 20.04

What's not working?

See above.

Job Log Output

N/A

Runner and Worker's Diagnostic Logs

N/A

We actually use that to our advantage (bad design, but no other solution) for mono repos to have required jobs per "directory" 🤦🏼‍♂️
The "faster" exit 0 jobs we use to exclude tooling folders.

[nedrebo]

We are also doing mono repo in GA. Silently allowing shadowing is making it hard to guarantee no regressions in the merge checks.

Another issue that may be relevant: #1115

[nedrebo]

I have added a reproducer here: nedrebo/github-actions-merge-check-go-green-prematurely#1

As you can see in the screenshot below the PR is unmergable after all jobs are finished. This is because the job called Delayed Failure / Required for merge did fail. This is as expected.

However, when the workflow was still running the situation was like this:

As you can see the merge button is activated even though the job called Delayed Failure / Required for merge has not yet started and as you can see from the first image it is "Required for merge". This is highly surprising behavior.

This bug can open up for fun times in a system that has merge bots and continuous delivery. I would consider this a critical bug.

[ruvceskistefan]

Hey all,

This issue does not seem to be a problem with the runner application, it concerns the GitHub actions platform more generally. Could you please post your feedback on the GitHub Community Support Forum which is actively monitored. Using the forum ensures that we route your problem to the correct team. 😃