Add a gatekeeper job for PR by cbeauchesne · Pull Request #8771 · DataDog/dd-trace-java

cbeauchesne · 2025-05-06T09:55:54Z

What Does This Do

Add a job in the CI that runs in PR. The job will be a success if all other jobs are skipped/success, and fails otherwise

Motivation

While it's possible to enforce a green CI policy using GitHub's native "required status checks" feature, doing so requires explicitly listing all job names under branch protection rules. This approach has two key drawbacks:

It does not support optional jobs
It introduces ongoing maintenance overhead as the job list evolves

This jobs will check ALL other job, and we'll be able to set this as a requirement. The action used offers a ignored-name-patterns parameters`, I added few job that failed more than 10% over the last 60 days on merges.

The plan is to merge this PR, wait few days to be sure that everything is fine. Then add it as a requirement.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-05-06T10:54:56Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/all-jobs-are-green
git_commit_date	1746535002	1746541857
git_commit_sha	`5bfab65`	`d5c1275`
release_version	1.49.0-SNAPSHOT~5bfab6549b	1.49.0-SNAPSHOT~d5c12752b6

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1746544804	1746544804
ci_job_id	925844837	925844837
ci_pipeline_id	64262684	64262684
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-mgrvxjgq-project-304-concurrent-0-6q589e2y 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-mgrvxjgq-project-304-concurrent-0-6q589e2y 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics.

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.009 s) : 0, 1008513
Total [baseline] (8.649 s) : 0, 8648950
Agent [candidate] (1.01 s) : 0, 1010093
Total [candidate] (8.635 s) : 0, 8634936
section iast
Agent [baseline] (1.137 s) : 0, 1136950
Total [baseline] (9.206 s) : 0, 9206160
Agent [candidate] (1.146 s) : 0, 1146297
Total [candidate] (9.227 s) : 0, 9226580
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.138 s) : 0, 1138292
Total [baseline] (9.213 s) : 0, 9212631
Agent [candidate] (1.145 s) : 0, 1144658
Total [candidate] (9.211 s) : 0, 9211237
section iast_TELEMETRY_OFF
Agent [baseline] (1.142 s) : 0, 1141655
Total [baseline] (9.209 s) : 0, 9209167
Agent [candidate] (1.133 s) : 0, 1133421
Total [candidate] (9.182 s) : 0, 9182088

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.009 s	-
Agent	iast	1.137 s	128.437 ms (12.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.138 s	129.779 ms (12.9%)
Agent	iast_TELEMETRY_OFF	1.142 s	133.142 ms (13.2%)
Total	tracing	8.649 s	-
Total	iast	9.206 s	557.21 ms (6.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.213 s	563.681 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.209 s	560.217 ms (6.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.01 s	-
Agent	iast	1.146 s	136.204 ms (13.5%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.145 s	134.565 ms (13.3%)
Agent	iast_TELEMETRY_OFF	1.133 s	123.328 ms (12.2%)
Total	tracing	8.635 s	-
Total	iast	9.227 s	591.644 ms (6.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.211 s	576.301 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.182 s	547.152 ms (6.3%)

gantt
title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.588 ms) : 0, 670588
BytebuddyAgent [candidate] (674.001 ms) : 0, 674001
GlobalTracer [baseline] (240.362 ms) : 0, 240362
GlobalTracer [candidate] (240.831 ms) : 0, 240831
AppSec [baseline] (54.588 ms) : 0, 54588
AppSec [candidate] (54.923 ms) : 0, 54923
Debugger [baseline] (7.645 ms) : 0, 7645
Debugger [candidate] (6.161 ms) : 0, 6161
Remote Config [baseline] (694.114 µs) : 0, 694
Remote Config [candidate] (705.119 µs) : 0, 705
Telemetry [baseline] (11.278 ms) : 0, 11278
Telemetry [candidate] (9.945 ms) : 0, 9945
section iast
BytebuddyAgent [baseline] (789.515 ms) : 0, 789515
BytebuddyAgent [candidate] (797.117 ms) : 0, 797117
GlobalTracer [baseline] (230.185 ms) : 0, 230185
GlobalTracer [candidate] (231.455 ms) : 0, 231455
IAST [baseline] (22.757 ms) : 0, 22757
IAST [candidate] (23.07 ms) : 0, 23070
AppSec [baseline] (56.578 ms) : 0, 56578
AppSec [candidate] (56.648 ms) : 0, 56648
Debugger [baseline] (5.926 ms) : 0, 5926
Debugger [candidate] (5.92 ms) : 0, 5920
Remote Config [baseline] (595.079 µs) : 0, 595
Remote Config [candidate] (585.843 µs) : 0, 586
Telemetry [baseline] (7.945 ms) : 0, 7945
Telemetry [candidate] (7.909 ms) : 0, 7909
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (790.733 ms) : 0, 790733
BytebuddyAgent [candidate] (795.769 ms) : 0, 795769
GlobalTracer [baseline] (230.481 ms) : 0, 230481
GlobalTracer [candidate] (231.354 ms) : 0, 231354
IAST [baseline] (22.776 ms) : 0, 22776
IAST [candidate] (23.076 ms) : 0, 23076
AppSec [baseline] (56.461 ms) : 0, 56461
AppSec [candidate] (56.47 ms) : 0, 56470
Debugger [baseline] (5.921 ms) : 0, 5921
Debugger [candidate] (5.92 ms) : 0, 5920
Remote Config [baseline] (610.643 µs) : 0, 611
Remote Config [candidate] (597.667 µs) : 0, 598
Telemetry [baseline] (7.888 ms) : 0, 7888
Telemetry [candidate] (7.895 ms) : 0, 7895
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.349 ms) : 0, 792349
BytebuddyAgent [candidate] (786.658 ms) : 0, 786658
GlobalTracer [baseline] (232.124 ms) : 0, 232124
GlobalTracer [candidate] (230.307 ms) : 0, 230307
IAST [baseline] (22.664 ms) : 0, 22664
IAST [candidate] (22.403 ms) : 0, 22403
AppSec [baseline] (56.505 ms) : 0, 56505
AppSec [candidate] (56.287 ms) : 0, 56287
Debugger [baseline] (5.981 ms) : 0, 5981
Debugger [candidate] (5.976 ms) : 0, 5976
Remote Config [baseline] (612.278 µs) : 0, 612
Remote Config [candidate] (600.254 µs) : 0, 600
Telemetry [baseline] (7.835 ms) : 0, 7835
Telemetry [candidate] (7.754 ms) : 0, 7754

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.009 s) : 0, 1008848
Total [baseline] (10.472 s) : 0, 10471920
Agent [candidate] (1.011 s) : 0, 1011101
Total [candidate] (10.533 s) : 0, 10532736
section appsec
Agent [baseline] (1.15 s) : 0, 1149539
Total [baseline] (10.709 s) : 0, 10708875
Agent [candidate] (1.153 s) : 0, 1153053
Total [candidate] (10.674 s) : 0, 10673959
section iast
Agent [baseline] (1.147 s) : 0, 1147231
Total [baseline] (10.941 s) : 0, 10941335
Agent [candidate] (1.139 s) : 0, 1139150
Total [candidate] (10.864 s) : 0, 10864312
section profiling
Agent [baseline] (1.261 s) : 0, 1261464
Total [baseline] (10.828 s) : 0, 10827875
Agent [candidate] (1.258 s) : 0, 1258026
Total [candidate] (10.801 s) : 0, 10800587

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.009 s	-
Agent	appsec	1.15 s	140.691 ms (13.9%)
Agent	iast	1.147 s	138.384 ms (13.7%)
Agent	profiling	1.261 s	252.617 ms (25.0%)
Total	tracing	10.472 s	-
Total	appsec	10.709 s	236.954 ms (2.3%)
Total	iast	10.941 s	469.415 ms (4.5%)
Total	profiling	10.828 s	355.954 ms (3.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.011 s	-
Agent	appsec	1.153 s	141.951 ms (14.0%)
Agent	iast	1.139 s	128.049 ms (12.7%)
Agent	profiling	1.258 s	246.925 ms (24.4%)
Total	tracing	10.533 s	-
Total	appsec	10.674 s	141.224 ms (1.3%)
Total	iast	10.864 s	331.576 ms (3.1%)
Total	profiling	10.801 s	267.851 ms (2.5%)

gantt
title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.977 ms) : 0, 671977
BytebuddyAgent [candidate] (674.329 ms) : 0, 674329
GlobalTracer [baseline] (240.403 ms) : 0, 240403
GlobalTracer [candidate] (241.209 ms) : 0, 241209
AppSec [baseline] (54.757 ms) : 0, 54757
AppSec [candidate] (55.062 ms) : 0, 55062
Debugger [baseline] (6.836 ms) : 0, 6836
Debugger [candidate] (6.195 ms) : 0, 6195
Remote Config [baseline] (693.513 µs) : 0, 694
Remote Config [candidate] (703.326 µs) : 0, 703
Telemetry [baseline] (10.677 ms) : 0, 10677
Telemetry [candidate] (10.053 ms) : 0, 10053
section appsec
BytebuddyAgent [baseline] (688.918 ms) : 0, 688918
BytebuddyAgent [candidate] (691.286 ms) : 0, 691286
GlobalTracer [baseline] (236.55 ms) : 0, 236550
GlobalTracer [candidate] (236.798 ms) : 0, 236798
IAST [baseline] (21.613 ms) : 0, 21613
IAST [candidate] (21.891 ms) : 0, 21891
AppSec [baseline] (175.9 ms) : 0, 175900
AppSec [candidate] (175.411 ms) : 0, 175411
Debugger [baseline] (5.874 ms) : 0, 5874
Debugger [candidate] (5.934 ms) : 0, 5934
Remote Config [baseline] (632.146 µs) : 0, 632
Remote Config [candidate] (648.135 µs) : 0, 648
Telemetry [baseline] (7.436 ms) : 0, 7436
Telemetry [candidate] (8.537 ms) : 0, 8537
section iast
BytebuddyAgent [baseline] (797.244 ms) : 0, 797244
BytebuddyAgent [candidate] (791.227 ms) : 0, 791227
GlobalTracer [baseline] (232.381 ms) : 0, 232381
GlobalTracer [candidate] (230.511 ms) : 0, 230511
IAST [baseline] (22.988 ms) : 0, 22988
IAST [candidate] (22.955 ms) : 0, 22955
AppSec [baseline] (56.626 ms) : 0, 56626
AppSec [candidate] (56.441 ms) : 0, 56441
Debugger [baseline] (5.928 ms) : 0, 5928
Debugger [candidate] (5.947 ms) : 0, 5947
Remote Config [baseline] (590.792 µs) : 0, 591
Remote Config [candidate] (605.926 µs) : 0, 606
Telemetry [baseline] (7.916 ms) : 0, 7916
Telemetry [candidate] (7.952 ms) : 0, 7952
section profiling
BytebuddyAgent [baseline] (664.284 ms) : 0, 664284
BytebuddyAgent [candidate] (662.639 ms) : 0, 662639
GlobalTracer [baseline] (381.102 ms) : 0, 381102
GlobalTracer [candidate] (379.098 ms) : 0, 379098
AppSec [baseline] (54.553 ms) : 0, 54553
AppSec [candidate] (54.087 ms) : 0, 54087
Debugger [baseline] (6.152 ms) : 0, 6152
Debugger [candidate] (6.152 ms) : 0, 6152
Remote Config [baseline] (668.417 µs) : 0, 668
Remote Config [candidate] (661.297 µs) : 0, 661
Telemetry [baseline] (8.238 ms) : 0, 8238
Telemetry [candidate] (8.179 ms) : 0, 8179
ProfilingAgent [baseline] (96.128 ms) : 0, 96128
ProfilingAgent [candidate] (97.053 ms) : 0, 97053
Profiling [baseline] (96.152 ms) : 0, 96152
Profiling [candidate] (97.076 ms) : 0, 97076

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-05-06T14:50:46	2025-05-06T14:58:29
git_branch	master	cbeauchesne/all-jobs-are-green
git_commit_date	1746535002	1746541857
git_commit_sha	`5bfab65`	`d5c1275`
release_version	1.49.0-SNAPSHOT~5bfab6549b	1.49.0-SNAPSHOT~d5c12752b6
start_time	2025-05-06T14:50:32	2025-05-06T14:58:14

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1746543904	1746543904
ci_job_id	925844838	925844838
ci_pipeline_id	64262684	64262684
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-drymq1by-project-304-concurrent-0-5hoguu4v 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-drymq1by-project-304-concurrent-0-5hoguu4v 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (1.359 ms) : 1339, 1378
. : milestone, 1359,
appsec (1.738 ms) : 1714, 1762
. : milestone, 1738,
appsec_no_iast (1.726 ms) : 1703, 1749
. : milestone, 1726,
code_origins (1.672 ms) : 1646, 1699
. : milestone, 1672,
iast (1.511 ms) : 1488, 1535
. : milestone, 1511,
profiling (1.502 ms) : 1479, 1525
. : milestone, 1502,
tracing (1.507 ms) : 1481, 1532
. : milestone, 1507,
section candidate
no_agent (1.348 ms) : 1328, 1367
. : milestone, 1348,
appsec (1.735 ms) : 1711, 1758
. : milestone, 1735,
appsec_no_iast (1.717 ms) : 1693, 1740
. : milestone, 1717,
code_origins (1.664 ms) : 1637, 1691
. : milestone, 1664,
iast (1.531 ms) : 1508, 1555
. : milestone, 1531,
profiling (1.482 ms) : 1459, 1505
. : milestone, 1482,
tracing (1.479 ms) : 1455, 1504
. : milestone, 1479,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.359 ms [1.339 ms, 1.378 ms]	-
appsec	1.738 ms [1.714 ms, 1.762 ms]	379.393 µs (27.9%)
appsec_no_iast	1.726 ms [1.703 ms, 1.749 ms]	367.557 µs (27.1%)
code_origins	1.672 ms [1.646 ms, 1.699 ms]	313.551 µs (23.1%)
iast	1.511 ms [1.488 ms, 1.535 ms]	152.545 µs (11.2%)
profiling	1.502 ms [1.479 ms, 1.525 ms]	143.305 µs (10.5%)
tracing	1.507 ms [1.481 ms, 1.532 ms]	147.789 µs (10.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.348 ms [1.328 ms, 1.367 ms]	-
appsec	1.735 ms [1.711 ms, 1.758 ms]	386.973 µs (28.7%)
appsec_no_iast	1.717 ms [1.693 ms, 1.74 ms]	368.94 µs (27.4%)
code_origins	1.664 ms [1.637 ms, 1.691 ms]	316.391 µs (23.5%)
iast	1.531 ms [1.508 ms, 1.555 ms]	183.408 µs (13.6%)
profiling	1.482 ms [1.459 ms, 1.505 ms]	134.41 µs (10.0%)
tracing	1.479 ms [1.455 ms, 1.504 ms]	131.702 µs (9.8%)

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (381.844 µs) : 362, 402
. : milestone, 382,
iast (520.101 µs) : 497, 543
. : milestone, 520,
iast_FULL (729.863 µs) : 706, 753
. : milestone, 730,
iast_GLOBAL (557.736 µs) : 534, 581
. : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (513.525 µs) : 491, 537
. : milestone, 514,
iast_INACTIVE (459.251 µs) : 437, 481
. : milestone, 459,
iast_TELEMETRY_OFF (502.823 µs) : 480, 526
. : milestone, 503,
tracing (459.319 µs) : 437, 481
. : milestone, 459,
section candidate
no_agent (376.032 µs) : 356, 396
. : milestone, 376,
iast (520.729 µs) : 498, 543
. : milestone, 521,
iast_FULL (728.278 µs) : 705, 752
. : milestone, 728,
iast_GLOBAL (575.529 µs) : 551, 600
. : milestone, 576,
iast_HARDCODED_SECRET_DISABLED (525.277 µs) : 503, 548
. : milestone, 525,
iast_INACTIVE (464.945 µs) : 442, 488
. : milestone, 465,
iast_TELEMETRY_OFF (500.544 µs) : 478, 523
. : milestone, 501,
tracing (460.736 µs) : 439, 483
. : milestone, 461,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.844 µs [361.805 µs, 401.883 µs]	-
iast	520.101 µs [497.093 µs, 543.11 µs]	138.257 µs (36.2%)
iast_FULL	729.863 µs [706.491 µs, 753.235 µs]	348.019 µs (91.1%)
iast_GLOBAL	557.736 µs [534.058 µs, 581.413 µs]	175.892 µs (46.1%)
iast_HARDCODED_SECRET_DISABLED	513.525 µs [490.538 µs, 536.512 µs]	131.681 µs (34.5%)
iast_INACTIVE	459.251 µs [437.376 µs, 481.127 µs]	77.407 µs (20.3%)
iast_TELEMETRY_OFF	502.823 µs [479.543 µs, 526.104 µs]	120.979 µs (31.7%)
tracing	459.319 µs [437.221 µs, 481.417 µs]	77.475 µs (20.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	376.032 µs [356.075 µs, 395.989 µs]	-
iast	520.729 µs [497.977 µs, 543.481 µs]	144.696 µs (38.5%)
iast_FULL	728.278 µs [704.943 µs, 751.612 µs]	352.245 µs (93.7%)
iast_GLOBAL	575.529 µs [551.2 µs, 599.859 µs]	199.497 µs (53.1%)
iast_HARDCODED_SECRET_DISABLED	525.277 µs [502.7 µs, 547.855 µs]	149.245 µs (39.7%)
iast_INACTIVE	464.945 µs [442.252 µs, 487.638 µs]	88.913 µs (23.6%)
iast_TELEMETRY_OFF	500.544 µs [477.752 µs, 523.336 µs]	124.512 µs (33.1%)
tracing	460.736 µs [438.519 µs, 482.954 µs]	84.704 µs (22.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/all-jobs-are-green
git_commit_date	1746535002	1746541857
git_commit_sha	`5bfab65`	`d5c1275`
release_version	1.49.0-SNAPSHOT~5bfab6549b	1.49.0-SNAPSHOT~d5c12752b6

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1746544405	1746544405
ci_job_id	925844839	925844839
ci_pipeline_id	64262684	64262684
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-mgrvxjgq-project-304-concurrent-1-4mbkvbe5 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-mgrvxjgq-project-304-concurrent-1-4mbkvbe5 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (14.976 s) : 14976000, 14976000
. : milestone, 14976000,
appsec (15.256 s) : 15256000, 15256000
. : milestone, 15256000,
iast (18.731 s) : 18731000, 18731000
. : milestone, 18731000,
iast_GLOBAL (17.968 s) : 17968000, 17968000
. : milestone, 17968000,
profiling (15.62 s) : 15620000, 15620000
. : milestone, 15620000,
tracing (14.817 s) : 14817000, 14817000
. : milestone, 14817000,
section candidate
no_agent (14.94 s) : 14940000, 14940000
. : milestone, 14940000,
appsec (14.938 s) : 14938000, 14938000
. : milestone, 14938000,
iast (18.822 s) : 18822000, 18822000
. : milestone, 18822000,
iast_GLOBAL (18.158 s) : 18158000, 18158000
. : milestone, 18158000,
profiling (15.021 s) : 15021000, 15021000
. : milestone, 15021000,
tracing (14.911 s) : 14911000, 14911000
. : milestone, 14911000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.976 s [14.976 s, 14.976 s]	-
appsec	15.256 s [15.256 s, 15.256 s]	280.0 ms (1.9%)
iast	18.731 s [18.731 s, 18.731 s]	3.755 s (25.1%)
iast_GLOBAL	17.968 s [17.968 s, 17.968 s]	2.992 s (20.0%)
profiling	15.62 s [15.62 s, 15.62 s]	644.0 ms (4.3%)
tracing	14.817 s [14.817 s, 14.817 s]	-159.0 ms (-1.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.94 s [14.94 s, 14.94 s]	-
appsec	14.938 s [14.938 s, 14.938 s]	-2.0 ms (-0.0%)
iast	18.822 s [18.822 s, 18.822 s]	3.882 s (26.0%)
iast_GLOBAL	18.158 s [18.158 s, 18.158 s]	3.218 s (21.5%)
profiling	15.021 s [15.021 s, 15.021 s]	81.0 ms (0.5%)
tracing	14.911 s [14.911 s, 14.911 s]	-29.0 ms (-0.2%)

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (2.388 ms) : 2341, 2436
. : milestone, 2388,
iast (2.173 ms) : 2113, 2232
. : milestone, 2173,
iast_GLOBAL (2.211 ms) : 2151, 2271
. : milestone, 2211,
profiling (2.047 ms) : 1998, 2097
. : milestone, 2047,
tracing (2.013 ms) : 1966, 2059
. : milestone, 2013,
section candidate
no_agent (1.477 ms) : 1466, 1489
. : milestone, 1477,
appsec (2.391 ms) : 2344, 2439
. : milestone, 2391,
iast (2.164 ms) : 2104, 2224
. : milestone, 2164,
iast_GLOBAL (2.213 ms) : 2153, 2274
. : milestone, 2213,
profiling (2.017 ms) : 1969, 2065
. : milestone, 2017,
tracing (1.993 ms) : 1946, 2039
. : milestone, 1993,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.465 ms, 1.488 ms]	-
appsec	2.388 ms [2.341 ms, 2.436 ms]	912.261 µs (61.8%)
iast	2.173 ms [2.113 ms, 2.232 ms]	696.514 µs (47.2%)
iast_GLOBAL	2.211 ms [2.151 ms, 2.271 ms]	735.088 µs (49.8%)
profiling	2.047 ms [1.998 ms, 2.097 ms]	571.185 µs (38.7%)
tracing	2.013 ms [1.966 ms, 2.059 ms]	536.78 µs (36.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.466 ms, 1.489 ms]	-
appsec	2.391 ms [2.344 ms, 2.439 ms]	913.892 µs (61.9%)
iast	2.164 ms [2.104 ms, 2.224 ms]	687.041 µs (46.5%)
iast_GLOBAL	2.213 ms [2.153 ms, 2.274 ms]	736.058 µs (49.8%)
profiling	2.017 ms [1.969 ms, 2.065 ms]	539.757 µs (36.5%)
tracing	1.993 ms [1.946 ms, 2.039 ms]	515.257 µs (34.9%)

PerfectSlayer

Interesting 🤔 That will solve the issue of having to craft GraphQL query to set branch protection rules.

This jobs will check ALL other job

What about the GitHub workflows? Are they checked too?
Or can we excluded them from the gateway and still requiring them from the GitHub protection rules (if they working fine already)?

And in general how does this work if the checks are not complete when the gatekeeper runs? Is that the retry part? Will it run for 40 mins?

This will be useful for #8475 cc @sarahchen6 @bric3

cbeauchesne · 2025-05-06T14:30:00Z

What about the GitHub workflows? Are they checked too?

Yes they are. By default, if you see something in your PR page, it should be checked.

Or can we excluded them from the gateway and still requiring them from the GitHub protection rules (if they working fine already)?

Yes you can, either by setting their names in ignored parameters, or by asking an option that does not exists already. But all the point of this is to avoid having a big list of required jobs, which is a pain to maintain, but rather a short list of excpetion (that should be by nature short).

And in general how does this work if the checks are not complete when the gatekeeper runs? Is that the retry part? Will it run for 40 mins?

Yes, there is a retry. With the current parameters :

 initial-delay-seconds: "1000"
max-retries: "60"
polling-interval-seconds: "60" (it's the default value)

So it runs for 1000 + 60*60 = 4600 seconds (1 hour and 10 minutes). Note that it perform an initial check at the very begining, to avoid waiting 1000s for nothing on retries.

PerfectSlayer · 2025-05-06T14:33:40Z

Thanks for the detailed answer! That looks handy 🙌

So it runs for 1000 + 60*60 = 4600 seconds (1 hour and 10 minutes).

That’s a lot! What about using the status event then? https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#status

cbeauchesne · 2025-05-06T14:40:37Z

That’s a lot! What about using the status event then?

Niiiiiiiice, I was not aware of that option! I need to perform proper verification to ensure it's a good fit. If you don't mind, I'll use system-tests repo to do the guinea pig, and if it's good, I'll be ack to use it here.

BTW, yes, it's a lot, but it will mostly stop before that, at your first failing job, or worst case scenario, your overall CI time.

PerfectSlayer

Sure, we can merge it as is, and iterate later if the new trigger work.

One thing I wonder is about the concurrency parameter. You might need to set it up in order to avoid having dozen of parallel runs for the same PR (which can trigger race condition if the last call -- with all the checks completed -- ends up before any of the previous one -- with not all the checks completed.

I will approve but I found can update the README file in the same folder about this automation / workflow, it could be nice (trigger, action, recovery) 👍

cbeauchesne · 2025-05-07T07:24:20Z

One thing I wonder is about the concurrency parameter.

Done in dcd3535

update the README file in the same folder

Done in b55b1f1

Add a gatekeeper job for PR
bbfd0c7

sarahchen6 added type: feature Enhancements and improvements comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes labels May 6, 2025

cbeauchesne marked this pull request as ready for review May 6, 2025 13:12

cbeauchesne requested a review from a team as a code owner May 6, 2025 13:12

cbeauchesne requested a review from smola May 6, 2025 13:12

PerfectSlayer reviewed May 6, 2025
View reviewed changes

smola reviewed May 6, 2025
View reviewed changes

Comment thread.github/workflows/all-green.yml Outdated

Include circleCI
d5c1275

PerfectSlayer approved these changes May 7, 2025
View reviewed changes

cbeauchesne added 2 commits May 7, 2025 09:18

Add concurrency
dcd3535

Add readme
b55b1f1

cbeauchesne enabled auto-merge (squash) May 7, 2025 07:24

cbeauchesne merged commit 1880ec1 into masterMay 7, 2025

cbeauchesne deleted the cbeauchesne/all-jobs-are-green branch May 7, 2025 07:53

github-actionsBot added this to the 1.50.0 milestone May 7, 2025

PerfectSlayer mentioned this pull request May 7, 2025
Remove muzzle from CircleCI #8703
Merged

Navigation Menu

Uh oh!

Add a gatekeeper job for PR - #8771

Add a gatekeeper job for PR#8771
cbeauchesne merged 4 commits into
masterfrom
cbeauchesne/all-jobs-are-green

cbeauchesne commented May 6, 2025

Uh oh!

pr-commenterBot commented May 6, 2025•
edited
Loading

Uh oh!

PerfectSlayer left a comment •
edited
Loading

Uh oh!

Uh oh!

cbeauchesne commented May 6, 2025

Uh oh!

PerfectSlayer commented May 6, 2025

Uh oh!

cbeauchesne commented May 6, 2025•
edited
Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

cbeauchesne commented May 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

cbeauchesne commented May 6, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented May 6, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cbeauchesne commented May 6, 2025

Uh oh!

PerfectSlayer commented May 6, 2025

Uh oh!

cbeauchesne commented May 6, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

cbeauchesne commented May 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pr-commenterBot commented May 6, 2025•
edited
Loading

PerfectSlayer left a comment •
edited
Loading

cbeauchesne commented May 6, 2025•
edited
Loading