Fixed flaky test by kr-igor · Pull Request #9218 · DataDog/dd-trace-java

kr-igor · 2025-07-22T17:42:37Z

What Does This Do

Fix for a flaky test

Details

Elements in the collection were not sorted properly. Now this test should work as expected.
The issue was introduced in this PR.

github-actions · 2025-07-22T17:43:03Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

AlexeyKuznetsov-DD

LGTM, do not forget to add labels and proper PR description by template

pr-commenter · 2025-07-22T18:16:34Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753206619	1753215515
git_commit_sha	`7b6259d`	`69efa01`
release_version	1.51.1-SNAPSHOT~7b6259d446	1.51.1-SNAPSHOT~69efa01716

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1753217451	1753217451
ci_job_id	1042184538	1042184538
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-gzbj0d6i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-gzbj0d6i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039345
Total [baseline] (10.69 s) : 0, 10689561
Agent [candidate] (1.04 s) : 0, 1039680
Total [candidate] (10.662 s) : 0, 10661526
section appsec
Agent [baseline] (1.223 s) : 0, 1222684
Total [baseline] (10.811 s) : 0, 10811109
Agent [candidate] (1.22 s) : 0, 1219539
Total [candidate] (10.756 s) : 0, 10756000
section iast
Agent [baseline] (1.173 s) : 0, 1173332
Total [baseline] (10.881 s) : 0, 10880865
Agent [candidate] (1.176 s) : 0, 1175671
Total [candidate] (10.905 s) : 0, 10905463
section profiling
Agent [baseline] (1.192 s) : 0, 1191762
Total [baseline] (10.899 s) : 0, 10899405
Agent [candidate] (1.185 s) : 0, 1184657
Total [candidate] (10.878 s) : 0, 10878138

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	appsec	1.223 s	183.339 ms (17.6%)
Agent	iast	1.173 s	133.986 ms (12.9%)
Agent	profiling	1.192 s	152.417 ms (14.7%)
Total	tracing	10.69 s	-
Total	appsec	10.811 s	121.548 ms (1.1%)
Total	iast	10.881 s	191.304 ms (1.8%)
Total	profiling	10.899 s	209.844 ms (2.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	appsec	1.22 s	179.859 ms (17.3%)
Agent	iast	1.176 s	135.991 ms (13.1%)
Agent	profiling	1.185 s	144.977 ms (13.9%)
Total	tracing	10.662 s	-
Total	appsec	10.756 s	94.474 ms (0.9%)
Total	iast	10.905 s	243.937 ms (2.3%)
Total	profiling	10.878 s	216.613 ms (2.0%)

gantt
title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.422 ms) : 0, 1422
crashtracking [candidate] (1.43 ms) : 0, 1430
BytebuddyAgent [baseline] (729.528 ms) : 0, 729528
BytebuddyAgent [candidate] (729.855 ms) : 0, 729855
GlobalTracer [baseline] (242.215 ms) : 0, 242215
GlobalTracer [candidate] (241.985 ms) : 0, 241985
AppSec [baseline] (30.372 ms) : 0, 30372
AppSec [candidate] (30.528 ms) : 0, 30528
Debugger [baseline] (5.982 ms) : 0, 5982
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (640.252 µs) : 0, 640
Remote Config [candidate] (642.719 µs) : 0, 643
Telemetry [baseline] (8.17 ms) : 0, 8170
Telemetry [candidate] (8.181 ms) : 0, 8181
section appsec
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (754.776 ms) : 0, 754776
BytebuddyAgent [candidate] (752.956 ms) : 0, 752956
GlobalTracer [baseline] (235.975 ms) : 0, 235975
GlobalTracer [candidate] (235.061 ms) : 0, 235061
AppSec [baseline] (168.865 ms) : 0, 168865
AppSec [candidate] (170.081 ms) : 0, 170081
Debugger [baseline] (7.216 ms) : 0, 7216
Debugger [candidate] (6.389 ms) : 0, 6389
Remote Config [baseline] (599.452 µs) : 0, 599
Remote Config [candidate] (599.61 µs) : 0, 600
Telemetry [baseline] (8.85 ms) : 0, 8850
Telemetry [candidate] (8.074 ms) : 0, 8074
IAST [baseline] (23.621 ms) : 0, 23621
IAST [candidate] (23.62 ms) : 0, 23620
section iast
crashtracking [baseline] (1.425 ms) : 0, 1425
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (846.909 ms) : 0, 846909
BytebuddyAgent [candidate] (848.898 ms) : 0, 848898
GlobalTracer [baseline] (232.247 ms) : 0, 232247
GlobalTracer [candidate] (231.636 ms) : 0, 231636
AppSec [baseline] (26.582 ms) : 0, 26582
AppSec [candidate] (29.114 ms) : 0, 29114
Debugger [baseline] (8.383 ms) : 0, 8383
Debugger [candidate] (6.706 ms) : 0, 6706
Remote Config [baseline] (589.623 µs) : 0, 590
Remote Config [candidate] (589.598 µs) : 0, 590
Telemetry [baseline] (7.895 ms) : 0, 7895
Telemetry [candidate] (7.985 ms) : 0, 7985
IAST [baseline] (28.298 ms) : 0, 28298
IAST [candidate] (28.186 ms) : 0, 28186
section profiling
ProfilingAgent [baseline] (105.765 ms) : 0, 105765
ProfilingAgent [candidate] (104.365 ms) : 0, 104365
crashtracking [baseline] (1.411 ms) : 0, 1411
crashtracking [candidate] (1.4 ms) : 0, 1400
BytebuddyAgent [baseline] (763.937 ms) : 0, 763937
BytebuddyAgent [candidate] (759.724 ms) : 0, 759724
GlobalTracer [baseline] (222.232 ms) : 0, 222232
GlobalTracer [candidate] (220.922 ms) : 0, 220922
AppSec [baseline] (30.751 ms) : 0, 30751
AppSec [candidate] (30.27 ms) : 0, 30270
Debugger [baseline] (6.338 ms) : 0, 6338
Debugger [candidate] (6.282 ms) : 0, 6282
Remote Config [baseline] (691.639 µs) : 0, 692
Remote Config [candidate] (666.293 µs) : 0, 666
Telemetry [baseline] (11.843 ms) : 0, 11843
Telemetry [candidate] (12.476 ms) : 0, 12476
Profiling [baseline] (105.789 ms) : 0, 105789
Profiling [candidate] (104.386 ms) : 0, 104386

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.04 s) : 0, 1040318
Total [baseline] (8.645 s) : 0, 8645309
Agent [candidate] (1.049 s) : 0, 1048660
Total [candidate] (8.619 s) : 0, 8618800
section iast
Agent [baseline] (1.173 s) : 0, 1172829
Total [baseline] (9.332 s) : 0, 9332105
Agent [candidate] (1.174 s) : 0, 1173597
Total [candidate] (9.272 s) : 0, 9271877

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	iast	1.173 s	132.51 ms (12.7%)
Total	tracing	8.645 s	-
Total	iast	9.332 s	686.796 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	iast	1.174 s	124.937 ms (11.9%)
Total	tracing	8.619 s	-
Total	iast	9.272 s	653.077 ms (7.6%)

gantt
title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.446 ms) : 0, 1446
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (731.107 ms) : 0, 731107
BytebuddyAgent [candidate] (735.029 ms) : 0, 735029
GlobalTracer [baseline] (241.568 ms) : 0, 241568
GlobalTracer [candidate] (244.444 ms) : 0, 244444
AppSec [baseline] (30.299 ms) : 0, 30299
AppSec [candidate] (30.842 ms) : 0, 30842
Debugger [baseline] (5.997 ms) : 0, 5997
Debugger [candidate] (6.066 ms) : 0, 6066
Remote Config [baseline] (638.038 µs) : 0, 638
Remote Config [candidate] (656.927 µs) : 0, 657
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (9.08 ms) : 0, 9080
section iast
crashtracking [baseline] (1.429 ms) : 0, 1429
crashtracking [candidate] (1.427 ms) : 0, 1427
BytebuddyAgent [baseline] (846.52 ms) : 0, 846520
BytebuddyAgent [candidate] (847.396 ms) : 0, 847396
GlobalTracer [baseline] (231.195 ms) : 0, 231195
GlobalTracer [candidate] (231.525 ms) : 0, 231525
AppSec [baseline] (27.46 ms) : 0, 27460
AppSec [candidate] (28.117 ms) : 0, 28117
Debugger [baseline] (8.346 ms) : 0, 8346
Debugger [candidate] (5.781 ms) : 0, 5781
Remote Config [baseline] (583.554 µs) : 0, 584
Remote Config [candidate] (584.698 µs) : 0, 585
Telemetry [baseline] (8.815 ms) : 0, 8815
Telemetry [candidate] (7.901 ms) : 0, 7901
IAST [baseline] (27.504 ms) : 0, 27504
IAST [candidate] (29.919 ms) : 0, 29919

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753206619	1753215515
git_commit_sha	`7b6259d`	`69efa01`
release_version	1.51.1-SNAPSHOT~7b6259d446	1.51.1-SNAPSHOT~69efa01716

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1753217197	1753217197
ci_job_id	1042184539	1042184539
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-bl6f9f1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-bl6f9f1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:appsec:high_load	better [-2.623ms; -1.695ms] or [-5.339%; -3.450%]	unstable [-2.635op/s; +11.310op/s] or [-2.766%; +11.873%]	46.972ms	99.600op/s	49.131ms	95.263op/s
scenario:load:petclinic:profiling:high_load	better [-2.693ms; -1.703ms] or [-5.449%; -3.445%]	unstable [-2.945op/s; +11.745op/s] or [-3.110%; +12.402%]	47.228ms	99.100op/s	49.426ms	94.700op/s
scenario:load:petclinic:tracing:high_load	worse [+1.126ms; +1.912ms] or [+2.610%; +4.434%]	unstable [-11.267op/s; +3.917op/s] or [-10.387%; +3.611%]	44.639ms	104.800op/s	43.120ms	108.475op/s
scenario:load:petclinic:code_origins:high_load	better [-2.146ms; -1.303ms] or [-4.651%; -2.824%]	unstable [-3.403op/s; +11.203op/s] or [-3.356%; +11.048%]	44.423ms	105.300op/s	46.148ms	101.400op/s

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section baseline
no_agent (36.955 ms) : 36649, 37261
. : milestone, 36955,
appsec (49.131 ms) : 48691, 49570
. : milestone, 49131,
code_origins (46.148 ms) : 45750, 46547
. : milestone, 46148,
iast (44.223 ms) : 43826, 44619
. : milestone, 44223,
profiling (49.426 ms) : 48952, 49900
. : milestone, 49426,
tracing (43.12 ms) : 42771, 43470
. : milestone, 43120,
section candidate
no_agent (37.047 ms) : 36755, 37338
. : milestone, 37047,
appsec (46.972 ms) : 46549, 47395
. : milestone, 46972,
code_origins (44.423 ms) : 44038, 44808
. : milestone, 44423,
iast (44.944 ms) : 44566, 45323
. : milestone, 44944,
profiling (47.228 ms) : 46782, 47675
. : milestone, 47228,
tracing (44.639 ms) : 44259, 45020
. : milestone, 44639,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.955 ms [36.649 ms, 37.261 ms]	-
appsec	49.131 ms [48.691 ms, 49.57 ms]	12.176 ms (32.9%)
code_origins	46.148 ms [45.75 ms, 46.547 ms]	9.193 ms (24.9%)
iast	44.223 ms [43.826 ms, 44.619 ms]	7.268 ms (19.7%)
profiling	49.426 ms [48.952 ms, 49.9 ms]	12.471 ms (33.7%)
tracing	43.12 ms [42.771 ms, 43.47 ms]	6.166 ms (16.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.047 ms [36.755 ms, 37.338 ms]	-
appsec	46.972 ms [46.549 ms, 47.395 ms]	9.925 ms (26.8%)
code_origins	44.423 ms [44.038 ms, 44.808 ms]	7.376 ms (19.9%)
iast	44.944 ms [44.566 ms, 45.323 ms]	7.898 ms (21.3%)
profiling	47.228 ms [46.782 ms, 47.675 ms]	10.181 ms (27.5%)
tracing	44.639 ms [44.259 ms, 45.02 ms]	7.593 ms (20.5%)

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section baseline
no_agent (4.336 ms) : 4275, 4396
. : milestone, 4336,
iast (9.571 ms) : 9402, 9740
. : milestone, 9571,
iast_FULL (13.921 ms) : 13646, 14197
. : milestone, 13921,
iast_GLOBAL (10.001 ms) : 9816, 10187
. : milestone, 10001,
profiling (9.019 ms) : 8876, 9161
. : milestone, 9019,
tracing (7.424 ms) : 7312, 7535
. : milestone, 7424,
section candidate
no_agent (4.375 ms) : 4322, 4428
. : milestone, 4375,
iast (9.474 ms) : 9313, 9636
. : milestone, 9474,
iast_FULL (13.813 ms) : 13539, 14086
. : milestone, 13813,
iast_GLOBAL (9.999 ms) : 9825, 10172
. : milestone, 9999,
profiling (9.316 ms) : 9174, 9459
. : milestone, 9316,
tracing (7.372 ms) : 7271, 7473
. : milestone, 7372,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.336 ms [4.275 ms, 4.396 ms]	-
iast	9.571 ms [9.402 ms, 9.74 ms]	5.235 ms (120.7%)
iast_FULL	13.921 ms [13.646 ms, 14.197 ms]	9.586 ms (221.1%)
iast_GLOBAL	10.001 ms [9.816 ms, 10.187 ms]	5.666 ms (130.7%)
profiling	9.019 ms [8.876 ms, 9.161 ms]	4.683 ms (108.0%)
tracing	7.424 ms [7.312 ms, 7.535 ms]	3.088 ms (71.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.375 ms [4.322 ms, 4.428 ms]	-
iast	9.474 ms [9.313 ms, 9.636 ms]	5.099 ms (116.6%)
iast_FULL	13.813 ms [13.539 ms, 14.086 ms]	9.437 ms (215.7%)
iast_GLOBAL	9.999 ms [9.825 ms, 10.172 ms]	5.624 ms (128.5%)
profiling	9.316 ms [9.174 ms, 9.459 ms]	4.941 ms (112.9%)
tracing	7.372 ms [7.271 ms, 7.473 ms]	2.997 ms (68.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753206619	1753215515
git_commit_sha	`7b6259d`	`69efa01`
release_version	1.51.1-SNAPSHOT~7b6259d446	1.51.1-SNAPSHOT~69efa01716

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1753217725	1753217725
ci_job_id	1042184540	1042184540
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-hs3nlbib 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-hs3nlbib 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
. : milestone, 1472,
appsec (3.616 ms) : 3401, 3831
. : milestone, 3616,
iast (2.187 ms) : 2125, 2250
. : milestone, 2187,
iast_GLOBAL (2.232 ms) : 2169, 2295
. : milestone, 2232,
profiling (2.053 ms) : 2001, 2105
. : milestone, 2053,
tracing (2.001 ms) : 1952, 2049
. : milestone, 2001,
section candidate
no_agent (1.476 ms) : 1464, 1488
. : milestone, 1476,
appsec (3.669 ms) : 3454, 3885
. : milestone, 3669,
iast (2.19 ms) : 2127, 2252
. : milestone, 2190,
iast_GLOBAL (2.25 ms) : 2187, 2314
. : milestone, 2250,
profiling (2.485 ms) : 2311, 2658
. : milestone, 2485,
tracing (2.008 ms) : 1959, 2057
. : milestone, 2008,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.46 ms, 1.483 ms]	-
appsec	3.616 ms [3.401 ms, 3.831 ms]	2.144 ms (145.7%)
iast	2.187 ms [2.125 ms, 2.25 ms]	715.569 µs (48.6%)
iast_GLOBAL	2.232 ms [2.169 ms, 2.295 ms]	760.056 µs (51.6%)
profiling	2.053 ms [2.001 ms, 2.105 ms]	581.849 µs (39.5%)
tracing	2.001 ms [1.952 ms, 2.049 ms]	529.247 µs (36.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.488 ms]	-
appsec	3.669 ms [3.454 ms, 3.885 ms]	2.193 ms (148.6%)
iast	2.19 ms [2.127 ms, 2.252 ms]	713.654 µs (48.3%)
iast_GLOBAL	2.25 ms [2.187 ms, 2.314 ms]	774.185 µs (52.5%)
profiling	2.485 ms [2.311 ms, 2.658 ms]	1.009 ms (68.3%)
tracing	2.008 ms [1.959 ms, 2.057 ms]	531.969 µs (36.0%)

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
dateFormat X
axisFormat %s
section baseline
no_agent (15.538 s) : 15538000, 15538000
. : milestone, 15538000,
appsec (14.937 s) : 14937000, 14937000
. : milestone, 14937000,
iast (18.011 s) : 18011000, 18011000
. : milestone, 18011000,
iast_GLOBAL (17.808 s) : 17808000, 17808000
. : milestone, 17808000,
profiling (15.36 s) : 15360000, 15360000
. : milestone, 15360000,
tracing (14.699 s) : 14699000, 14699000
. : milestone, 14699000,
section candidate
no_agent (15.503 s) : 15503000, 15503000
. : milestone, 15503000,
appsec (14.991 s) : 14991000, 14991000
. : milestone, 14991000,
iast (18.875 s) : 18875000, 18875000
. : milestone, 18875000,
iast_GLOBAL (18.006 s) : 18006000, 18006000
. : milestone, 18006000,
profiling (15.573 s) : 15573000, 15573000
. : milestone, 15573000,
tracing (14.927 s) : 14927000, 14927000
. : milestone, 14927000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.538 s [15.538 s, 15.538 s]	-
appsec	14.937 s [14.937 s, 14.937 s]	-601.0 ms (-3.9%)
iast	18.011 s [18.011 s, 18.011 s]	2.473 s (15.9%)
iast_GLOBAL	17.808 s [17.808 s, 17.808 s]	2.27 s (14.6%)
profiling	15.36 s [15.36 s, 15.36 s]	-178.0 ms (-1.1%)
tracing	14.699 s [14.699 s, 14.699 s]	-839.0 ms (-5.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.503 s [15.503 s, 15.503 s]	-
appsec	14.991 s [14.991 s, 14.991 s]	-512.0 ms (-3.3%)
iast	18.875 s [18.875 s, 18.875 s]	3.372 s (21.8%)
iast_GLOBAL	18.006 s [18.006 s, 18.006 s]	2.503 s (16.1%)
profiling	15.573 s [15.573 s, 15.573 s]	70.0 ms (0.5%)
tracing	14.927 s [14.927 s, 14.927 s]	-576.0 ms (-3.7%)

pr-commenter · 2025-07-22T18:18:25Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753205644	1753215515
git_commit_sha	`f21ceea`	`69efa01`

See matching parameters

	Baseline	Candidate
ci_job_date	1753216787	1753216787
ci_job_id	1042184543	1042184543
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

pr-commenter · 2025-07-22T21:10:16Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753205644	1753215515
git_commit_sha	`f21ceea`	`69efa01`

See matching parameters

	Baseline	Candidate
ci_job_date	1753216787	1753216787
ci_job_id	1042184544	1042184544
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

Fixed flaky test
557cbfc

kr-igor marked this pull request as ready for review July 22, 2025 17:42

kr-igor requested a review from a team as a code owner July 22, 2025 17:42

kr-igor added comp: testing Testing comp: data streams Data Streams Monitoring labels Jul 22, 2025

AlexeyKuznetsov-DD approved these changes Jul 22, 2025
View reviewed changes

kr-igor added the type: bug fix Bug fix label Jul 22, 2025

bm1549 mentioned this pull request Jul 22, 2025
Match Hands Off Config selectors on process_arguments value #9201
Merged

AlexeyKuznetsov-DD added the tag: no release notes Changes to exclude from release notes label Jul 22, 2025

Removed kafka cluster id check, as it's sometimes equal to empty string
69efa01

kr-igor merged commit 2469447 into masterJul 22, 2025
505 checks passed

kr-igor deleted the kr-igor/fix-flaky-test branch July 22, 2025 21:33

github-actionsBot added this to the 1.52.0 milestone Jul 22, 2025

Navigation Menu

Uh oh!

Fixed flaky test - #9218

Fixed flaky test#9218
kr-igor merged 2 commits into
masterfrom
kr-igor/fix-flaky-test

kr-igor commented Jul 22, 2025•
edited
Loading

Uh oh!

github-actionsBot commented Jul 22, 2025•
edited
Loading

Uh oh!

AlexeyKuznetsov-DD left a comment

Uh oh!

pr-commenterBot commented Jul 22, 2025•
edited
Loading

Uh oh!

pr-commenterBot commented Jul 22, 2025•
edited
Loading

Uh oh!

pr-commenterBot commented Jul 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kr-igor commented Jul 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Details

Uh oh!

github-actionsBot commented Jul 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

Uh oh!

pr-commenterBot commented Jul 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenterBot commented Jul 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenterBot commented Jul 22, 2025

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kr-igor commented Jul 22, 2025•
edited
Loading

github-actionsBot commented Jul 22, 2025•
edited
Loading

pr-commenterBot commented Jul 22, 2025•
edited
Loading

pr-commenterBot commented Jul 22, 2025•
edited
Loading