Migrate benchmarks to Merge Queue by sarahchen6 · Pull Request #10526 · DataDog/dd-trace-java

sarahchen6 · 2026-02-04T20:29:05Z

What Does This Do

Run startup, load, and dacapo benchmarks on the merge queue instead of for every PR push. Allow these benchmarks to be run manually in the corresponding Gitlab pipeline as well.

Motivation

By migrating these benchmarks to the merge queue, we can greatly reduce CI runtime (from ~60 min to ~30 min) and support faster development experiences. At the same time, however, we don't want to lose benchmarking ability nor merge in unknown regressions. This is addressed by still allowing manual benchmark runs on PRs and the check-big-regressions gate.

Additional Notes

The check-big-regressions check will still be in place and fail on regressions of 10+%; however, it will not catch small or unstable regressions. Developers should still manually run benchmarks once done iterating to see results.

See this 30 min pipeline run with benchmarks on MQ, and this 52 min pipeline run with manually run benchmarks.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2026-04-29T16:59:12Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-benchmarks-to-mq
git_commit_date	1777473948	1777475289
git_commit_sha	`c9cebad`	`3585ce7`
release_version	1.62.0-SNAPSHOT~c9cebadc2b	1.62.0-SNAPSHOT~3585ce7875

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1777477667	1777477667
ci_job_id	1642801752	1642801752
ci_pipeline_id	110468322	110468322
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-z514h0zm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-z514h0zm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1082691
Total [baseline] (11.03 s) : 0, 11030017
Agent [candidate] (1.078 s) : 0, 1077995
Total [candidate] (11.091 s) : 0, 11090702
section appsec
Agent [baseline] (1.271 s) : 0, 1270637
Total [baseline] (11.131 s) : 0, 11130979
Agent [candidate] (1.268 s) : 0, 1267576
Total [candidate] (11.103 s) : 0, 11103184
section iast
Agent [baseline] (1.256 s) : 0, 1256357
Total [baseline] (11.254 s) : 0, 11254386
Agent [candidate] (1.245 s) : 0, 1245354
Total [candidate] (11.252 s) : 0, 11251783
section profiling
Agent [baseline] (1.2 s) : 0, 1200348
Total [baseline] (11.128 s) : 0, 11127935
Agent [candidate] (1.188 s) : 0, 1188494
Total [candidate] (11.064 s) : 0, 11064281

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	appsec	1.271 s	187.947 ms (17.4%)
Agent	iast	1.256 s	173.667 ms (16.0%)
Agent	profiling	1.2 s	117.658 ms (10.9%)
Total	tracing	11.03 s	-
Total	appsec	11.131 s	100.962 ms (0.9%)
Total	iast	11.254 s	224.369 ms (2.0%)
Total	profiling	11.128 s	97.918 ms (0.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.078 s	-
Agent	appsec	1.268 s	189.581 ms (17.6%)
Agent	iast	1.245 s	167.36 ms (15.5%)
Agent	profiling	1.188 s	110.5 ms (10.3%)
Total	tracing	11.091 s	-
Total	appsec	11.103 s	12.482 ms (0.1%)
Total	iast	11.252 s	161.081 ms (1.5%)
Total	profiling	11.064 s	-26.42 ms (-0.2%)

gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.246 ms) : 0, 1246
crashtracking [candidate] (1.234 ms) : 0, 1234
BytebuddyAgent [baseline] (645.626 ms) : 0, 645626
BytebuddyAgent [candidate] (644.54 ms) : 0, 644540
AgentMeter [baseline] (30.001 ms) : 0, 30001
AgentMeter [candidate] (29.774 ms) : 0, 29774
GlobalTracer [baseline] (252.623 ms) : 0, 252623
GlobalTracer [candidate] (250.237 ms) : 0, 250237
AppSec [baseline] (33.203 ms) : 0, 33203
AppSec [candidate] (33.032 ms) : 0, 33032
Debugger [baseline] (63.053 ms) : 0, 63053
Debugger [candidate] (62.296 ms) : 0, 62296
Remote Config [baseline] (619.146 µs) : 0, 619
Remote Config [candidate] (621.606 µs) : 0, 622
Telemetry [baseline] (8.551 ms) : 0, 8551
Telemetry [candidate] (9.202 ms) : 0, 9202
Flare Poller [baseline] (11.465 ms) : 0, 11465
Flare Poller [candidate] (10.689 ms) : 0, 10689
section appsec
crashtracking [baseline] (1.235 ms) : 0, 1235
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (677.46 ms) : 0, 677460
BytebuddyAgent [candidate] (675.213 ms) : 0, 675213
AgentMeter [baseline] (12.355 ms) : 0, 12355
AgentMeter [candidate] (12.208 ms) : 0, 12208
GlobalTracer [baseline] (249.828 ms) : 0, 249828
GlobalTracer [candidate] (249.341 ms) : 0, 249341
AppSec [baseline] (185.275 ms) : 0, 185275
AppSec [candidate] (185.224 ms) : 0, 185224
Debugger [baseline] (65.986 ms) : 0, 65986
Debugger [candidate] (66.006 ms) : 0, 66006
Remote Config [baseline] (574.115 µs) : 0, 574
Remote Config [candidate] (578.526 µs) : 0, 579
Telemetry [baseline] (7.835 ms) : 0, 7835
Telemetry [candidate] (7.851 ms) : 0, 7851
Flare Poller [baseline] (8.61 ms) : 0, 8610
Flare Poller [candidate] (6.857 ms) : 0, 6857
IAST [baseline] (24.717 ms) : 0, 24717
IAST [candidate] (24.621 ms) : 0, 24621
section iast
crashtracking [baseline] (1.238 ms) : 0, 1238
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (830.272 ms) : 0, 830272
BytebuddyAgent [candidate] (822.871 ms) : 0, 822871
AgentMeter [baseline] (11.397 ms) : 0, 11397
AgentMeter [candidate] (11.299 ms) : 0, 11299
GlobalTracer [baseline] (240.131 ms) : 0, 240131
GlobalTracer [candidate] (237.861 ms) : 0, 237861
AppSec [baseline] (30.302 ms) : 0, 30302
AppSec [candidate] (31.37 ms) : 0, 31370
Debugger [baseline] (66.244 ms) : 0, 66244
Debugger [candidate] (64.562 ms) : 0, 64562
Remote Config [baseline] (538.973 µs) : 0, 539
Remote Config [candidate] (523.754 µs) : 0, 524
Telemetry [baseline] (7.984 ms) : 0, 7984
Telemetry [candidate] (7.931 ms) : 0, 7931
Flare Poller [baseline] (3.413 ms) : 0, 3413
Flare Poller [candidate] (3.42 ms) : 0, 3420
IAST [baseline] (27.641 ms) : 0, 27641
IAST [candidate] (28.221 ms) : 0, 28221
section profiling
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (700.913 ms) : 0, 700913
BytebuddyAgent [candidate] (692.226 ms) : 0, 692226
AgentMeter [baseline] (8.978 ms) : 0, 8978
AgentMeter [candidate] (8.937 ms) : 0, 8937
GlobalTracer [baseline] (209.247 ms) : 0, 209247
GlobalTracer [candidate] (207.85 ms) : 0, 207850
AppSec [baseline] (33.132 ms) : 0, 33132
AppSec [candidate] (32.665 ms) : 0, 32665
Debugger [baseline] (68.156 ms) : 0, 68156
Debugger [candidate] (67.566 ms) : 0, 67566
Remote Config [baseline] (584.948 µs) : 0, 585
Remote Config [candidate] (563.626 µs) : 0, 564
Telemetry [baseline] (8.123 ms) : 0, 8123
Telemetry [candidate] (8.146 ms) : 0, 8146
Flare Poller [baseline] (3.556 ms) : 0, 3556
Flare Poller [candidate] (3.569 ms) : 0, 3569
ProfilingAgent [baseline] (94.136 ms) : 0, 94136
ProfilingAgent [candidate] (94.17 ms) : 0, 94170
Profiling [baseline] (94.693 ms) : 0, 94693
Profiling [candidate] (94.727 ms) : 0, 94727

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1073284
Total [baseline] (8.857 s) : 0, 8856510
Agent [candidate] (1.066 s) : 0, 1066316
Total [candidate] (8.844 s) : 0, 8843956
section iast
Agent [baseline] (1.254 s) : 0, 1254069
Total [baseline] (9.561 s) : 0, 9561331
Agent [candidate] (1.243 s) : 0, 1243491
Total [candidate] (9.5 s) : 0, 9500118

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.073 s	-
Agent	iast	1.254 s	180.786 ms (16.8%)
Total	tracing	8.857 s	-
Total	iast	9.561 s	704.821 ms (8.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	iast	1.243 s	177.175 ms (16.6%)
Total	tracing	8.844 s	-
Total	iast	9.5 s	656.162 ms (7.4%)

gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.24 ms) : 0, 1240
crashtracking [candidate] (1.23 ms) : 0, 1230
BytebuddyAgent [baseline] (641.126 ms) : 0, 641126
BytebuddyAgent [candidate] (636.537 ms) : 0, 636537
AgentMeter [baseline] (29.6 ms) : 0, 29600
AgentMeter [candidate] (29.483 ms) : 0, 29483
GlobalTracer [baseline] (249.984 ms) : 0, 249984
GlobalTracer [candidate] (249.356 ms) : 0, 249356
AppSec [baseline] (32.999 ms) : 0, 32999
AppSec [candidate] (32.69 ms) : 0, 32690
Debugger [baseline] (61.639 ms) : 0, 61639
Debugger [candidate] (61.468 ms) : 0, 61468
Remote Config [baseline] (604.642 µs) : 0, 605
Remote Config [candidate] (598.755 µs) : 0, 599
Telemetry [baseline] (9.891 ms) : 0, 9891
Telemetry [candidate] (8.339 ms) : 0, 8339
Flare Poller [baseline] (10.038 ms) : 0, 10038
Flare Poller [candidate] (10.677 ms) : 0, 10677
section iast
crashtracking [baseline] (1.242 ms) : 0, 1242
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (830.519 ms) : 0, 830519
BytebuddyAgent [candidate] (822.182 ms) : 0, 822182
AgentMeter [baseline] (11.411 ms) : 0, 11411
AgentMeter [candidate] (11.28 ms) : 0, 11280
GlobalTracer [baseline] (238.205 ms) : 0, 238205
GlobalTracer [candidate] (237.469 ms) : 0, 237469
AppSec [baseline] (31.669 ms) : 0, 31669
AppSec [candidate] (30.528 ms) : 0, 30528
Debugger [baseline] (64.541 ms) : 0, 64541
Debugger [candidate] (64.692 ms) : 0, 64692
Remote Config [baseline] (524.905 µs) : 0, 525
Remote Config [candidate] (517.39 µs) : 0, 517
Telemetry [baseline] (7.826 ms) : 0, 7826
Telemetry [candidate] (7.926 ms) : 0, 7926
Flare Poller [baseline] (3.367 ms) : 0, 3367
Flare Poller [candidate] (3.341 ms) : 0, 3341
IAST [baseline] (28.553 ms) : 0, 28553
IAST [candidate] (28.274 ms) : 0, 28274

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-benchmarks-to-mq
git_commit_date	1777473948	1777475289
git_commit_sha	`c9cebad`	`3585ce7`
release_version	1.62.0-SNAPSHOT~c9cebadc2b	1.62.0-SNAPSHOT~3585ce7875

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1777477963	1777477963
ci_job_id	1642801754	1642801754
ci_pipeline_id	110468322	110468322
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-3-ilcfu6o6 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-3-ilcfu6o6 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 17 metrics, 18 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:appsec:high_load	better [-1.918ms; -0.859ms] or [-9.641%; -4.319%]	unsure [-2.548ms; -0.472ms] or [-8.061%; -1.493%]	unstable [-11.108op/s; +40.170op/s] or [-4.775%; +17.268%]	18.504ms	30.100ms	247.156op/s	19.893ms	31.610ms	232.625op/s

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section baseline
no_agent (1.183 ms) : 1171, 1194
. : milestone, 1183,
iast (3.283 ms) : 3235, 3331
. : milestone, 3283,
iast_FULL (5.83 ms) : 5771, 5889
. : milestone, 5830,
iast_GLOBAL (3.594 ms) : 3524, 3663
. : milestone, 3594,
profiling (2.126 ms) : 2107, 2145
. : milestone, 2126,
tracing (1.849 ms) : 1835, 1864
. : milestone, 1849,
section candidate
no_agent (1.237 ms) : 1226, 1249
. : milestone, 1237,
iast (3.29 ms) : 3242, 3338
. : milestone, 3290,
iast_FULL (5.983 ms) : 5922, 6044
. : milestone, 5983,
iast_GLOBAL (3.657 ms) : 3594, 3721
. : milestone, 3657,
profiling (2.155 ms) : 2135, 2175
. : milestone, 2155,
tracing (1.868 ms) : 1853, 1883
. : milestone, 1868,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.183 ms [1.171 ms, 1.194 ms]	-
iast	3.283 ms [3.235 ms, 3.331 ms]	2.101 ms (177.6%)
iast_FULL	5.83 ms [5.771 ms, 5.889 ms]	4.647 ms (393.0%)
iast_GLOBAL	3.594 ms [3.524 ms, 3.663 ms]	2.411 ms (203.9%)
profiling	2.126 ms [2.107 ms, 2.145 ms]	943.399 µs (79.8%)
tracing	1.849 ms [1.835 ms, 1.864 ms]	666.901 µs (56.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.237 ms [1.226 ms, 1.249 ms]	-
iast	3.29 ms [3.242 ms, 3.338 ms]	2.053 ms (166.0%)
iast_FULL	5.983 ms [5.922 ms, 6.044 ms]	4.746 ms (383.6%)
iast_GLOBAL	3.657 ms [3.594 ms, 3.721 ms]	2.42 ms (195.6%)
profiling	2.155 ms [2.135 ms, 2.175 ms]	918.157 µs (74.2%)
tracing	1.868 ms [1.853 ms, 1.883 ms]	630.98 µs (51.0%)

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section baseline
no_agent (18.208 ms) : 18023, 18393
. : milestone, 18208,
appsec (20.069 ms) : 19862, 20275
. : milestone, 20069,
code_origins (17.946 ms) : 17769, 18124
. : milestone, 17946,
iast (17.769 ms) : 17593, 17945
. : milestone, 17769,
profiling (18.701 ms) : 18512, 18890
. : milestone, 18701,
tracing (17.805 ms) : 17630, 17981
. : milestone, 17805,
section candidate
no_agent (18.665 ms) : 18471, 18858
. : milestone, 18665,
appsec (18.883 ms) : 18688, 19078
. : milestone, 18883,
code_origins (18.041 ms) : 17865, 18217
. : milestone, 18041,
iast (17.811 ms) : 17638, 17984
. : milestone, 17811,
profiling (18.24 ms) : 18062, 18417
. : milestone, 18240,
tracing (17.846 ms) : 17667, 18026
. : milestone, 17846,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.208 ms [18.023 ms, 18.393 ms]	-
appsec	20.069 ms [19.862 ms, 20.275 ms]	1.861 ms (10.2%)
code_origins	17.946 ms [17.769 ms, 18.124 ms]	-261.549 µs (-1.4%)
iast	17.769 ms [17.593 ms, 17.945 ms]	-438.472 µs (-2.4%)
profiling	18.701 ms [18.512 ms, 18.89 ms]	493.166 µs (2.7%)
tracing	17.805 ms [17.63 ms, 17.981 ms]	-402.473 µs (-2.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.665 ms [18.471 ms, 18.858 ms]	-
appsec	18.883 ms [18.688 ms, 19.078 ms]	218.637 µs (1.2%)
code_origins	18.041 ms [17.865 ms, 18.217 ms]	-624.025 µs (-3.3%)
iast	17.811 ms [17.638 ms, 17.984 ms]	-853.333 µs (-4.6%)
profiling	18.24 ms [18.062 ms, 18.417 ms]	-425.037 µs (-2.3%)
tracing	17.846 ms [17.667 ms, 18.026 ms]	-818.22 µs (-4.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-benchmarks-to-mq
git_commit_date	1777473948	1777475289
git_commit_sha	`c9cebad`	`3585ce7`
release_version	1.62.0-SNAPSHOT~c9cebadc2b	1.62.0-SNAPSHOT~3585ce7875

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1777477646	1777477646
ci_job_id	1642801755	1642801755
ci_pipeline_id	110468322	110468322
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-j7wouqpg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-j7wouqpg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section baseline
no_agent (1.49 ms) : 1478, 1502
. : milestone, 1490,
appsec (3.842 ms) : 3619, 4064
. : milestone, 3842,
iast (2.28 ms) : 2210, 2350
. : milestone, 2280,
iast_GLOBAL (2.338 ms) : 2268, 2409
. : milestone, 2338,
profiling (2.103 ms) : 2048, 2157
. : milestone, 2103,
tracing (2.098 ms) : 2044, 2152
. : milestone, 2098,
section candidate
no_agent (1.495 ms) : 1483, 1507
. : milestone, 1495,
appsec (3.832 ms) : 3608, 4057
. : milestone, 3832,
iast (2.282 ms) : 2212, 2352
. : milestone, 2282,
iast_GLOBAL (2.325 ms) : 2255, 2395
. : milestone, 2325,
profiling (2.106 ms) : 2051, 2161
. : milestone, 2106,
tracing (2.087 ms) : 2034, 2141
. : milestone, 2087,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.49 ms [1.478 ms, 1.502 ms]	-
appsec	3.842 ms [3.619 ms, 4.064 ms]	2.351 ms (157.8%)
iast	2.28 ms [2.21 ms, 2.35 ms]	790.303 µs (53.0%)
iast_GLOBAL	2.338 ms [2.268 ms, 2.409 ms]	848.312 µs (56.9%)
profiling	2.103 ms [2.048 ms, 2.157 ms]	612.486 µs (41.1%)
tracing	2.098 ms [2.044 ms, 2.152 ms]	607.662 µs (40.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.495 ms [1.483 ms, 1.507 ms]	-
appsec	3.832 ms [3.608 ms, 4.057 ms]	2.337 ms (156.3%)
iast	2.282 ms [2.212 ms, 2.352 ms]	786.763 µs (52.6%)
iast_GLOBAL	2.325 ms [2.255 ms, 2.395 ms]	829.629 µs (55.5%)
profiling	2.106 ms [2.051 ms, 2.161 ms]	611.195 µs (40.9%)
tracing	2.087 ms [2.034 ms, 2.141 ms]	592.151 µs (39.6%)

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~3585ce7875, baseline=1.62.0-SNAPSHOT~c9cebadc2b
dateFormat X
axisFormat %s
section baseline
no_agent (14.874 s) : 14874000, 14874000
. : milestone, 14874000,
appsec (14.946 s) : 14946000, 14946000
. : milestone, 14946000,
iast (19.831 s) : 19831000, 19831000
. : milestone, 19831000,
iast_GLOBAL (17.95 s) : 17950000, 17950000
. : milestone, 17950000,
profiling (14.999 s) : 14999000, 14999000
. : milestone, 14999000,
tracing (15.018 s) : 15018000, 15018000
. : milestone, 15018000,
section candidate
no_agent (15.979 s) : 15979000, 15979000
. : milestone, 15979000,
appsec (14.892 s) : 14892000, 14892000
. : milestone, 14892000,
iast (18.651 s) : 18651000, 18651000
. : milestone, 18651000,
iast_GLOBAL (17.911 s) : 17911000, 17911000
. : milestone, 17911000,
profiling (14.948 s) : 14948000, 14948000
. : milestone, 14948000,
tracing (14.919 s) : 14919000, 14919000
. : milestone, 14919000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.874 s [14.874 s, 14.874 s]	-
appsec	14.946 s [14.946 s, 14.946 s]	72.0 ms (0.5%)
iast	19.831 s [19.831 s, 19.831 s]	4.957 s (33.3%)
iast_GLOBAL	17.95 s [17.95 s, 17.95 s]	3.076 s (20.7%)
profiling	14.999 s [14.999 s, 14.999 s]	125.0 ms (0.8%)
tracing	15.018 s [15.018 s, 15.018 s]	144.0 ms (1.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.979 s [15.979 s, 15.979 s]	-
appsec	14.892 s [14.892 s, 14.892 s]	-1.087 s (-6.8%)
iast	18.651 s [18.651 s, 18.651 s]	2.672 s (16.7%)
iast_GLOBAL	17.911 s [17.911 s, 17.911 s]	1.932 s (12.1%)
profiling	14.948 s [14.948 s, 14.948 s]	-1.031 s (-6.5%)
tracing	14.919 s [14.919 s, 14.919 s]	-1.06 s (-6.6%)

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:3585ce7875

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

sarahchen6 · 2026-04-29T18:24:23Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-04-29T18:24:26Z

View all feedbacks in Devflow UI.

2026-04-29 18:24:26 UTC ℹ️ Start processing command /merge

2026-04-29 18:24:31 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-04-29 19:37:10 UTC ℹ️ MergeQueue: This merge request was merged

sarahchen6 added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Feb 4, 2026

Run dacapo/load/startup benchmarks on merge queue and manually
32b3af7

sarahchen6 force-pushed the sarahchen6/move-benchmarks-to-mq branch from c3c9539 to 32b3af7Compare April 28, 2026 19:27

Apply to parallel benchmarks as well
61774a0

sarahchen6 changed the title ~~Migrate benchmarks to MergeQueue~~Migrate benchmarks to Merge QueueApr 29, 2026

Merge branch 'master' into sarahchen6/move-benchmarks-to-mq
3585ce7

sarahchen6 marked this pull request as ready for review April 29, 2026 17:07

sarahchen6 requested a review from a team as a code owner April 29, 2026 17:07

sarahchen6 requested review from AlexeyKuznetsov-DD and randomanderson and removed request for a teamApril 29, 2026 17:07

chatgpt-codex-connectorBot reviewed Apr 29, 2026
View reviewed changes

Comment thread.gitlab/benchmarks.yml Outdated

AlexeyKuznetsov-DD approved these changes Apr 29, 2026
View reviewed changes

Fix slash
06708b4

gh-worker-dd-mergequeue-cf854dBot merged commit 52c2ac6 into masterApr 29, 2026
567 of 568 checks passed

gh-worker-dd-mergequeue-cf854dBot deleted the sarahchen6/move-benchmarks-to-mq branch April 29, 2026 19:37

github-actionsBot added this to the 1.62.0 milestone Apr 29, 2026

Navigation Menu

Uh oh!

Migrate benchmarks to Merge Queue - #10526

Migrate benchmarks to Merge Queue#10526
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
masterfrom
sarahchen6/move-benchmarks-to-mq

sarahchen6 commented Feb 4, 2026•
edited
Loading

Uh oh!

pr-commenterBot commented Apr 29, 2026

Uh oh!

chatgpt-codex-connectorBot left a comment

Uh oh!

Uh oh!

sarahchen6 commented Apr 29, 2026

Uh oh!

gh-worker-devflow-routing-ef8351Bot commented Apr 29, 2026•
edited
Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

sarahchen6 commented Feb 4, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Apr 29, 2026

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

sarahchen6 commented Apr 29, 2026

Uh oh!

gh-worker-devflow-routing-ef8351Bot commented Apr 29, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sarahchen6 commented Feb 4, 2026•
edited
Loading

gh-worker-devflow-routing-ef8351Bot commented Apr 29, 2026•
edited
Loading