Various Gitlab tweaks and test fixes by randomanderson · Pull Request #8847 · DataDog/dd-trace-java

randomanderson · 2025-05-19T14:15:53Z

What Does This Do

Bump memory for tests
Increase memory for forked tests
Fix ExceptionHistogramTest on J9

Motivation

I was regularly seeing OOMs on forked tests when running on all JDKs (the nightly). The 512mb limit is lower than the limit for regular tests. Some of these OOMs were nonobvious because the job would fail much later

This PR aligns forks to use 1024mb across all the ways tests could be forked.

Additionally, I fixed the ExceptionHistogramTest, which only failed during the nightly.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

jpbempel

LGTM for debugger part

pr-commenter · 2025-05-19T14:18:57Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	landerson/gitlab-test-fixes
git_commit_date	1747655134	1747661920
git_commit_sha	`6a77487`	`78bc339`
release_version	1.50.0-SNAPSHOT~6a77487ae5	1.50.0-SNAPSHOT~78bc339fc4

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747664289	1747664289
ci_job_id	943822207	943822207
ci_pipeline_id	65481444	65481444
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-6ae23wjf-project-304-concurrent-0-cajlhulb 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-6ae23wjf-project-304-concurrent-0-cajlhulb 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 15 unstable metrics.

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.023 s) : 0, 1022633
Total [baseline] (8.669 s) : 0, 8669343
Agent [candidate] (1.02 s) : 0, 1020420
Total [candidate] (8.659 s) : 0, 8659307
section iast
Agent [baseline] (1.157 s) : 0, 1157115
Total [baseline] (9.263 s) : 0, 9262719
Agent [candidate] (1.153 s) : 0, 1152656
Total [candidate] (9.226 s) : 0, 9226446
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.15 s) : 0, 1150178
Total [baseline] (9.199 s) : 0, 9198887
Agent [candidate] (1.157 s) : 0, 1157107
Total [candidate] (9.253 s) : 0, 9253062
section iast_TELEMETRY_OFF
Agent [baseline] (1.144 s) : 0, 1143929
Total [baseline] (9.255 s) : 0, 9255429
Agent [candidate] (1.162 s) : 0, 1162376
Total [candidate] (9.305 s) : 0, 9305126

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.023 s	-
Agent	iast	1.157 s	134.482 ms (13.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.15 s	127.545 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.144 s	121.296 ms (11.9%)
Total	tracing	8.669 s	-
Total	iast	9.263 s	593.376 ms (6.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.199 s	529.545 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.255 s	586.087 ms (6.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.02 s	-
Agent	iast	1.153 s	132.236 ms (13.0%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.157 s	136.686 ms (13.4%)
Agent	iast_TELEMETRY_OFF	1.162 s	141.955 ms (13.9%)
Total	tracing	8.659 s	-
Total	iast	9.226 s	567.139 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.253 s	593.754 ms (6.9%)
Total	iast_TELEMETRY_OFF	9.305 s	645.819 ms (7.5%)

gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.184 ms) : 0, 685184
BytebuddyAgent [candidate] (681.951 ms) : 0, 681951
GlobalTracer [baseline] (238.669 ms) : 0, 238669
GlobalTracer [candidate] (239.849 ms) : 0, 239849
AppSec [baseline] (55.039 ms) : 0, 55039
AppSec [candidate] (54.536 ms) : 0, 54536
Debugger [baseline] (9.743 ms) : 0, 9743
Debugger [candidate] (9.131 ms) : 0, 9131
Remote Config [baseline] (689.663 µs) : 0, 690
Remote Config [candidate] (695.971 µs) : 0, 696
Telemetry [baseline] (9.742 ms) : 0, 9742
Telemetry [candidate] (10.672 ms) : 0, 10672
section iast
BytebuddyAgent [baseline] (807.695 ms) : 0, 807695
BytebuddyAgent [candidate] (803.975 ms) : 0, 803975
GlobalTracer [baseline] (231.973 ms) : 0, 231973
GlobalTracer [candidate] (231.935 ms) : 0, 231935
IAST [baseline] (27.844 ms) : 0, 27844
IAST [candidate] (29.407 ms) : 0, 29407
AppSec [baseline] (51.508 ms) : 0, 51508
AppSec [candidate] (48.7 ms) : 0, 48700
Debugger [baseline] (5.962 ms) : 0, 5962
Debugger [candidate] (5.887 ms) : 0, 5887
Remote Config [baseline] (594.899 µs) : 0, 595
Remote Config [candidate] (600.405 µs) : 0, 600
Telemetry [baseline] (7.939 ms) : 0, 7939
Telemetry [candidate] (7.92 ms) : 0, 7920
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (802.638 ms) : 0, 802638
BytebuddyAgent [candidate] (807.259 ms) : 0, 807259
GlobalTracer [baseline] (230.884 ms) : 0, 230884
GlobalTracer [candidate] (231.941 ms) : 0, 231941
IAST [baseline] (30.028 ms) : 0, 30028
IAST [candidate] (28.818 ms) : 0, 28818
AppSec [baseline] (48.699 ms) : 0, 48699
AppSec [candidate] (50.781 ms) : 0, 50781
Debugger [baseline] (5.892 ms) : 0, 5892
Debugger [candidate] (6.006 ms) : 0, 6006
Remote Config [baseline] (582.978 µs) : 0, 583
Remote Config [candidate] (612.679 µs) : 0, 613
Telemetry [baseline] (7.977 ms) : 0, 7977
Telemetry [candidate] (8.044 ms) : 0, 8044
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (797.614 ms) : 0, 797614
BytebuddyAgent [candidate] (810.891 ms) : 0, 810891
GlobalTracer [baseline] (230.276 ms) : 0, 230276
GlobalTracer [candidate] (233.422 ms) : 0, 233422
IAST [baseline] (23.08 ms) : 0, 23080
IAST [candidate] (24.184 ms) : 0, 24184
AppSec [baseline] (55.173 ms) : 0, 55173
AppSec [candidate] (55.247 ms) : 0, 55247
Debugger [baseline] (5.959 ms) : 0, 5959
Debugger [candidate] (6.116 ms) : 0, 6116
Remote Config [baseline] (607.119 µs) : 0, 607
Remote Config [candidate] (640.102 µs) : 0, 640
Telemetry [baseline] (7.773 ms) : 0, 7773
Telemetry [candidate] (8.005 ms) : 0, 8005

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.017 s) : 0, 1017296
Total [baseline] (10.5 s) : 0, 10500341
Agent [candidate] (1.022 s) : 0, 1021861
Total [candidate] (10.506 s) : 0, 10505565
section appsec
Agent [baseline] (1.165 s) : 0, 1164919
Total [baseline] (10.673 s) : 0, 10672926
Agent [candidate] (1.16 s) : 0, 1160164
Total [candidate] (10.663 s) : 0, 10663073
section iast
Agent [baseline] (1.156 s) : 0, 1156319
Total [baseline] (10.953 s) : 0, 10952572
Agent [candidate] (1.157 s) : 0, 1157104
Total [candidate] (11.095 s) : 0, 11095010
section profiling
Agent [baseline] (1.296 s) : 0, 1295514
Total [baseline] (10.917 s) : 0, 10916701
Agent [candidate] (1.282 s) : 0, 1281785
Total [candidate] (10.854 s) : 0, 10853772

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.017 s	-
Agent	appsec	1.165 s	147.622 ms (14.5%)
Agent	iast	1.156 s	139.023 ms (13.7%)
Agent	profiling	1.296 s	278.218 ms (27.3%)
Total	tracing	10.5 s	-
Total	appsec	10.673 s	172.586 ms (1.6%)
Total	iast	10.953 s	452.231 ms (4.3%)
Total	profiling	10.917 s	416.36 ms (4.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.022 s	-
Agent	appsec	1.16 s	138.303 ms (13.5%)
Agent	iast	1.157 s	135.243 ms (13.2%)
Agent	profiling	1.282 s	259.923 ms (25.4%)
Total	tracing	10.506 s	-
Total	appsec	10.663 s	157.508 ms (1.5%)
Total	iast	11.095 s	589.445 ms (5.6%)
Total	profiling	10.854 s	348.207 ms (3.3%)

gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.241 ms) : 0, 682241
BytebuddyAgent [candidate] (682.942 ms) : 0, 682942
GlobalTracer [baseline] (239.813 ms) : 0, 239813
GlobalTracer [candidate] (240.55 ms) : 0, 240550
AppSec [baseline] (54.244 ms) : 0, 54244
AppSec [candidate] (54.489 ms) : 0, 54489
Debugger [baseline] (6.886 ms) : 0, 6886
Debugger [candidate] (9.145 ms) : 0, 9145
Remote Config [baseline] (716.331 µs) : 0, 716
Remote Config [candidate] (705.548 µs) : 0, 706
Telemetry [baseline] (9.845 ms) : 0, 9845
Telemetry [candidate] (10.429 ms) : 0, 10429
section appsec
BytebuddyAgent [baseline] (702.941 ms) : 0, 702941
BytebuddyAgent [candidate] (699.926 ms) : 0, 699926
GlobalTracer [baseline] (237.372 ms) : 0, 237372
GlobalTracer [candidate] (236.355 ms) : 0, 236355
AppSec [baseline] (176.284 ms) : 0, 176284
AppSec [candidate] (175.233 ms) : 0, 175233
Debugger [baseline] (5.926 ms) : 0, 5926
Debugger [candidate] (5.925 ms) : 0, 5925
Remote Config [baseline] (633.916 µs) : 0, 634
Remote Config [candidate] (623.08 µs) : 0, 623
Telemetry [baseline] (7.356 ms) : 0, 7356
Telemetry [candidate] (7.755 ms) : 0, 7755
IAST [baseline] (21.679 ms) : 0, 21679
IAST [candidate] (21.686 ms) : 0, 21686
section iast
BytebuddyAgent [baseline] (807.411 ms) : 0, 807411
BytebuddyAgent [candidate] (805.54 ms) : 0, 805540
GlobalTracer [baseline] (232.332 ms) : 0, 232332
GlobalTracer [candidate] (233.614 ms) : 0, 233614
AppSec [baseline] (48.613 ms) : 0, 48613
AppSec [candidate] (51.03 ms) : 0, 51030
Debugger [baseline] (5.888 ms) : 0, 5888
Debugger [candidate] (5.981 ms) : 0, 5981
Remote Config [baseline] (592.509 µs) : 0, 593
Remote Config [candidate] (605.463 µs) : 0, 605
Telemetry [baseline] (7.872 ms) : 0, 7872
Telemetry [candidate] (7.99 ms) : 0, 7990
IAST [baseline] (29.962 ms) : 0, 29962
IAST [candidate] (28.814 ms) : 0, 28814
section profiling
ProfilingAgent [baseline] (103.721 ms) : 0, 103721
ProfilingAgent [candidate] (103.212 ms) : 0, 103212
BytebuddyAgent [baseline] (683.605 ms) : 0, 683605
BytebuddyAgent [candidate] (674.705 ms) : 0, 674705
GlobalTracer [baseline] (379.165 ms) : 0, 379165
GlobalTracer [candidate] (375.209 ms) : 0, 375209
AppSec [baseline] (62.396 ms) : 0, 62396
AppSec [candidate] (62.536 ms) : 0, 62536
Debugger [baseline] (6.288 ms) : 0, 6288
Debugger [candidate] (6.252 ms) : 0, 6252
Remote Config [baseline] (673.646 µs) : 0, 674
Remote Config [candidate] (651.958 µs) : 0, 652
Telemetry [baseline] (8.278 ms) : 0, 8278
Telemetry [candidate] (8.264 ms) : 0, 8264
Profiling [baseline] (103.745 ms) : 0, 103745
Profiling [candidate] (103.236 ms) : 0, 103236

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-05-19T13:50:18	2025-05-19T13:58:01
git_branch	master	landerson/gitlab-test-fixes
git_commit_date	1747655134	1747661920
git_commit_sha	`6a77487`	`78bc339`
release_version	1.50.0-SNAPSHOT~6a77487ae5	1.50.0-SNAPSHOT~78bc339fc4
start_time	2025-05-19T13:50:04	2025-05-19T13:57:47

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747663479	1747663479
ci_job_id	943822209	943822209
ci_pipeline_id	65481444	65481444
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-boakpwxa-project-304-concurrent-0-4ij3w3d9 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-boakpwxa-project-304-concurrent-0-4ij3w3d9 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 19 unstable metrics.

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section baseline
no_agent (382.909 µs) : 363, 403
. : milestone, 383,
iast (523.292 µs) : 502, 545
. : milestone, 523,
iast_FULL (734.032 µs) : 712, 756
. : milestone, 734,
iast_GLOBAL (565.507 µs) : 544, 587
. : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (524.37 µs) : 502, 547
. : milestone, 524,
iast_INACTIVE (469.669 µs) : 447, 493
. : milestone, 470,
iast_TELEMETRY_OFF (512.342 µs) : 488, 537
. : milestone, 512,
tracing (465.282 µs) : 443, 488
. : milestone, 465,
section candidate
no_agent (386.284 µs) : 367, 406
. : milestone, 386,
iast (517.82 µs) : 496, 540
. : milestone, 518,
iast_FULL (738.55 µs) : 716, 761
. : milestone, 739,
iast_GLOBAL (569.78 µs) : 547, 592
. : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (525.194 µs) : 503, 548
. : milestone, 525,
iast_INACTIVE (471.151 µs) : 449, 493
. : milestone, 471,
iast_TELEMETRY_OFF (508.779 µs) : 486, 532
. : milestone, 509,
tracing (459.617 µs) : 438, 481
. : milestone, 460,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	382.909 µs [363.164 µs, 402.654 µs]	-
iast	523.292 µs [501.603 µs, 544.982 µs]	140.384 µs (36.7%)
iast_FULL	734.032 µs [712.245 µs, 755.819 µs]	351.124 µs (91.7%)
iast_GLOBAL	565.507 µs [543.821 µs, 587.194 µs]	182.599 µs (47.7%)
iast_HARDCODED_SECRET_DISABLED	524.37 µs [502.113 µs, 546.628 µs]	141.462 µs (36.9%)
iast_INACTIVE	469.669 µs [446.818 µs, 492.521 µs]	86.761 µs (22.7%)
iast_TELEMETRY_OFF	512.342 µs [487.931 µs, 536.753 µs]	129.433 µs (33.8%)
tracing	465.282 µs [442.994 µs, 487.569 µs]	82.373 µs (21.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	386.284 µs [366.795 µs, 405.774 µs]	-
iast	517.82 µs [495.978 µs, 539.662 µs]	131.536 µs (34.1%)
iast_FULL	738.55 µs [716.468 µs, 760.631 µs]	352.265 µs (91.2%)
iast_GLOBAL	569.78 µs [547.477 µs, 592.082 µs]	183.495 µs (47.5%)
iast_HARDCODED_SECRET_DISABLED	525.194 µs [502.752 µs, 547.636 µs]	138.909 µs (36.0%)
iast_INACTIVE	471.151 µs [448.985 µs, 493.316 µs]	84.866 µs (22.0%)
iast_TELEMETRY_OFF	508.779 µs [485.544 µs, 532.015 µs]	122.495 µs (31.7%)
tracing	459.617 µs [437.772 µs, 481.461 µs]	73.332 µs (19.0%)

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section baseline
no_agent (1.357 ms) : 1337, 1377
. : milestone, 1357,
appsec (1.743 ms) : 1720, 1766
. : milestone, 1743,
appsec_no_iast (1.72 ms) : 1697, 1743
. : milestone, 1720,
code_origins (1.678 ms) : 1651, 1706
. : milestone, 1678,
iast (1.508 ms) : 1484, 1531
. : milestone, 1508,
profiling (1.496 ms) : 1472, 1520
. : milestone, 1496,
tracing (1.496 ms) : 1471, 1520
. : milestone, 1496,
section candidate
no_agent (1.374 ms) : 1354, 1394
. : milestone, 1374,
appsec (1.729 ms) : 1706, 1752
. : milestone, 1729,
appsec_no_iast (1.738 ms) : 1715, 1761
. : milestone, 1738,
code_origins (1.679 ms) : 1652, 1706
. : milestone, 1679,
iast (1.515 ms) : 1492, 1539
. : milestone, 1515,
profiling (1.524 ms) : 1501, 1548
. : milestone, 1524,
tracing (1.497 ms) : 1472, 1521
. : milestone, 1497,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.357 ms [1.337 ms, 1.377 ms]	-
appsec	1.743 ms [1.72 ms, 1.766 ms]	385.896 µs (28.4%)
appsec_no_iast	1.72 ms [1.697 ms, 1.743 ms]	362.833 µs (26.7%)
code_origins	1.678 ms [1.651 ms, 1.706 ms]	320.967 µs (23.6%)
iast	1.508 ms [1.484 ms, 1.531 ms]	150.228 µs (11.1%)
profiling	1.496 ms [1.472 ms, 1.52 ms]	138.826 µs (10.2%)
tracing	1.496 ms [1.471 ms, 1.52 ms]	138.374 µs (10.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.374 ms [1.354 ms, 1.394 ms]	-
appsec	1.729 ms [1.706 ms, 1.752 ms]	354.857 µs (25.8%)
appsec_no_iast	1.738 ms [1.715 ms, 1.761 ms]	364.006 µs (26.5%)
code_origins	1.679 ms [1.652 ms, 1.706 ms]	304.944 µs (22.2%)
iast	1.515 ms [1.492 ms, 1.539 ms]	141.337 µs (10.3%)
profiling	1.524 ms [1.501 ms, 1.548 ms]	150.451 µs (11.0%)
tracing	1.497 ms [1.472 ms, 1.521 ms]	122.582 µs (8.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	landerson/gitlab-test-fixes
git_commit_date	1747655134	1747661920
git_commit_sha	`6a77487`	`78bc339`
release_version	1.50.0-SNAPSHOT~6a77487ae5	1.50.0-SNAPSHOT~78bc339fc4

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1747663826	1747663826
ci_job_id	943822211	943822211
ci_pipeline_id	65481444	65481444
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-6ae23wjf-project-304-concurrent-1-gys1e68a 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-6ae23wjf-project-304-concurrent-1-gys1e68a 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section baseline
no_agent (15.237 s) : 15237000, 15237000
. : milestone, 15237000,
appsec (15.182 s) : 15182000, 15182000
. : milestone, 15182000,
iast (18.605 s) : 18605000, 18605000
. : milestone, 18605000,
iast_GLOBAL (18.045 s) : 18045000, 18045000
. : milestone, 18045000,
profiling (15.482 s) : 15482000, 15482000
. : milestone, 15482000,
tracing (14.84 s) : 14840000, 14840000
. : milestone, 14840000,
section candidate
no_agent (15.304 s) : 15304000, 15304000
. : milestone, 15304000,
appsec (15.086 s) : 15086000, 15086000
. : milestone, 15086000,
iast (19.202 s) : 19202000, 19202000
. : milestone, 19202000,
iast_GLOBAL (17.983 s) : 17983000, 17983000
. : milestone, 17983000,
profiling (15.208 s) : 15208000, 15208000
. : milestone, 15208000,
tracing (15.046 s) : 15046000, 15046000
. : milestone, 15046000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.237 s [15.237 s, 15.237 s]	-
appsec	15.182 s [15.182 s, 15.182 s]	-55.0 ms (-0.4%)
iast	18.605 s [18.605 s, 18.605 s]	3.368 s (22.1%)
iast_GLOBAL	18.045 s [18.045 s, 18.045 s]	2.808 s (18.4%)
profiling	15.482 s [15.482 s, 15.482 s]	245.0 ms (1.6%)
tracing	14.84 s [14.84 s, 14.84 s]	-397.0 ms (-2.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.304 s [15.304 s, 15.304 s]	-
appsec	15.086 s [15.086 s, 15.086 s]	-218.0 ms (-1.4%)
iast	19.202 s [19.202 s, 19.202 s]	3.898 s (25.5%)
iast_GLOBAL	17.983 s [17.983 s, 17.983 s]	2.679 s (17.5%)
profiling	15.208 s [15.208 s, 15.208 s]	-96.0 ms (-0.6%)
tracing	15.046 s [15.046 s, 15.046 s]	-258.0 ms (-1.7%)

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~78bc339fc4, baseline=1.50.0-SNAPSHOT~6a77487ae5
dateFormat X
axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
. : milestone, 1478,
appsec (2.41 ms) : 2361, 2459
. : milestone, 2410,
iast (2.194 ms) : 2133, 2256
. : milestone, 2194,
iast_GLOBAL (2.241 ms) : 2179, 2303
. : milestone, 2241,
profiling (2.03 ms) : 1980, 2079
. : milestone, 2030,
tracing (2.014 ms) : 1966, 2062
. : milestone, 2014,
section candidate
no_agent (1.476 ms) : 1464, 1487
. : milestone, 1476,
appsec (2.41 ms) : 2361, 2459
. : milestone, 2410,
iast (2.188 ms) : 2126, 2250
. : milestone, 2188,
iast_GLOBAL (2.237 ms) : 2175, 2300
. : milestone, 2237,
profiling (2.067 ms) : 2016, 2119
. : milestone, 2067,
tracing (2.034 ms) : 1985, 2082
. : milestone, 2034,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	2.41 ms [2.361 ms, 2.459 ms]	932.416 µs (63.1%)
iast	2.194 ms [2.133 ms, 2.256 ms]	716.683 µs (48.5%)
iast_GLOBAL	2.241 ms [2.179 ms, 2.303 ms]	763.586 µs (51.7%)
profiling	2.03 ms [1.98 ms, 2.079 ms]	551.906 µs (37.4%)
tracing	2.014 ms [1.966 ms, 2.062 ms]	536.119 µs (36.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.41 ms [2.361 ms, 2.459 ms]	934.352 µs (63.3%)
iast	2.188 ms [2.126 ms, 2.25 ms]	712.123 µs (48.3%)
iast_GLOBAL	2.237 ms [2.175 ms, 2.3 ms]	761.304 µs (51.6%)
profiling	2.067 ms [2.016 ms, 2.119 ms]	591.191 µs (40.1%)
tracing	2.034 ms [1.985 ms, 2.082 ms]	558.0 µs (37.8%)

bric3 · 2025-05-19T15:38:57Z

 fi
 - *prepare_test_env
- - export GRADLE_OPTS="-Dorg.gradle.jvmargs='-Xms$GRADLE_MEM -Xmx$GRADLE_MEM $PROFILER_COMMAND -XX:ErrorFile=/tmp/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp' -Ddatadog.forkedMaxHeapSize=768M -Ddatadog.forkedMinHeapSize=128M"
+ - export GRADLE_OPTS="-Dorg.gradle.jvmargs='-Xms$GRADLE_MEM -Xmx$GRADLE_MEM $PROFILER_COMMAND -XX:ErrorFile=/tmp/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp' -Ddatadog.forkedMaxHeapSize=1024M -Ddatadog.forkedMinHeapSize=128M"


suggestion: Maybe make the forkedMaxHeapSize a variable.

bric3 · 2025-05-19T15:42:30Z

 fi
 - *prepare_test_env
- - export GRADLE_OPTS="-Dorg.gradle.jvmargs='-Xms$GRADLE_MEM -Xmx$GRADLE_MEM $PROFILER_COMMAND -XX:ErrorFile=/tmp/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp' -Ddatadog.forkedMaxHeapSize=768M -Ddatadog.forkedMinHeapSize=128M"
+ - export GRADLE_OPTS="-Dorg.gradle.jvmargs='-Xms$GRADLE_MEM -Xmx$GRADLE_MEM $PROFILER_COMMAND -XX:ErrorFile=/tmp/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp' -Ddatadog.forkedMaxHeapSize=1024M -Ddatadog.forkedMinHeapSize=128M"


suggestion: On another topic, I believe we should remove -Xms param here (or reduce the minimum like 512M or 1G). I removed some during the CircleCi tweaks

* skip should be for entire class * bump memory to combat oom issues * bump smoketests and forked tests memory and defaults

randomanderson added 3 commits May 16, 2025 14:41

skip should be for entire class
289f86e

bump memory to combat oom issues
d080a37

bump smoketests and forked tests memory and defaults
78bc339

randomanderson requested review from a team as code owners May 19, 2025 14:15

randomanderson added the tag: no release notes Changes to exclude from release notes label May 19, 2025

randomanderson requested a review from a team as a code owner May 19, 2025 14:15

randomanderson added the comp: tooling Build & Tooling label May 19, 2025

randomanderson requested review from amarziali, pawelchcki and shatzi and removed request for a teamMay 19, 2025 14:15

jpbempel approved these changes May 19, 2025
View reviewed changes

bric3 approved these changes May 19, 2025
View reviewed changes

randomanderson enabled auto-merge (squash) May 20, 2025 09:08

randomanderson merged commit dec7833 into masterMay 20, 2025

randomanderson deleted the landerson/gitlab-test-fixes branch May 20, 2025 10:12

github-actionsBot added this to the 1.50.0 milestone May 20, 2025

sarahchen6 pushed a commit that referenced this pull request Jun 12, 2025
Various Gitlab tweaks and test fixes (#8847)
8cd5fa8
* skip should be for entire class * bump memory to combat oom issues * bump smoketests and forked tests memory and defaults

Navigation Menu

Uh oh!

Various Gitlab tweaks and test fixes - #8847

Various Gitlab tweaks and test fixes#8847
randomanderson merged 3 commits into
masterfrom
landerson/gitlab-test-fixes

randomanderson commented May 19, 2025

Uh oh!

jpbempel left a comment

Uh oh!

pr-commenterBot commented May 19, 2025

Uh oh!

bric3May 19, 2025

Uh oh!

bric3May 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

randomanderson commented May 19, 2025

What Does This Do

Motivation

Contributor Checklist

Uh oh!

jpbempel left a comment

Choose a reason for hiding this comment

Uh oh!

pr-commenterBot commented May 19, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

bric3May 19, 2025

Choose a reason for hiding this comment

Uh oh!

bric3May 19, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants