Fix Maven smoke test timeout in CI by bric3 · Pull Request #8993 · DataDog/dd-trace-java

bric3 · 2025-06-16T14:27:48Z

What Does This Do

Increases the maven smoke test timeout for CI. Numerous maven smoke tests are failing in CI, du to process timeout. Locally the tests are passing, let's increase a tad the timeouts.

java.util.concurrent.TimeoutException: Instrumented process failed to exit
at datadog.smoketest.MavenSmokeTest.runProcess(MavenSmokeTest.groovy:336)
at datadog.smoketest.MavenSmokeTest.retryUntilSuccessfulOrNoAttemptsLeft(MavenSmokeTest.groovy:312)
at datadog.smoketest.MavenSmokeTest.givenMavenDependenciesAreLoaded(MavenSmokeTest.groovy:294)
at datadog.smoketest.MavenSmokeTest.test #projectName, v#mavenVersion(MavenSmokeTest.groovy:65)

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-06-16T15:01:28Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/increasemaven-smoke-test-timeout
git_commit_date	1750076840	1750083912
git_commit_sha	`9db71d4`	`9efa172`
release_version	1.50.0-SNAPSHOT~9db71d4352	1.50.0-SNAPSHOT~9efa17209f

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750086037	1750086037
ci_job_id	983615637	983615637
ci_pipeline_id	67851301	67851301
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-z87jatgw-project-304-concurrent-0-dg6a855z 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-z87jatgw-project-304-concurrent-0-dg6a855z 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics.

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1030228
Total [baseline] (8.537 s) : 0, 8536842
Agent [candidate] (1.027 s) : 0, 1027087
Total [candidate] (8.527 s) : 0, 8526713
section iast
Agent [baseline] (1.156 s) : 0, 1156352
Total [baseline] (9.215 s) : 0, 9215031
Agent [candidate] (1.162 s) : 0, 1162034
Total [candidate] (9.268 s) : 0, 9267711
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.164 s) : 0, 1163926
Total [baseline] (9.182 s) : 0, 9181700
Agent [candidate] (1.157 s) : 0, 1156742
Total [candidate] (9.214 s) : 0, 9214170
section iast_TELEMETRY_OFF
Agent [baseline] (1.15 s) : 0, 1150062
Total [baseline] (9.208 s) : 0, 9207710
Agent [candidate] (1.155 s) : 0, 1154535
Total [candidate] (9.199 s) : 0, 9198904

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	iast	1.156 s	126.124 ms (12.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.164 s	133.698 ms (13.0%)
Agent	iast_TELEMETRY_OFF	1.15 s	119.834 ms (11.6%)
Total	tracing	8.537 s	-
Total	iast	9.215 s	678.189 ms (7.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.182 s	644.858 ms (7.6%)
Total	iast_TELEMETRY_OFF	9.208 s	670.868 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	iast	1.162 s	134.947 ms (13.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.157 s	129.655 ms (12.6%)
Agent	iast_TELEMETRY_OFF	1.155 s	127.448 ms (12.4%)
Total	tracing	8.527 s	-
Total	iast	9.268 s	740.998 ms (8.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.214 s	687.458 ms (8.1%)
Total	iast_TELEMETRY_OFF	9.199 s	672.191 ms (7.9%)

gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.22 ms) : 0, 685220
BytebuddyAgent [candidate] (683.884 ms) : 0, 683884
GlobalTracer [baseline] (242.53 ms) : 0, 242530
GlobalTracer [candidate] (241.287 ms) : 0, 241287
AppSec [baseline] (59.705 ms) : 0, 59705
AppSec [candidate] (59.337 ms) : 0, 59337
Debugger [baseline] (6.228 ms) : 0, 6228
Debugger [candidate] (6.228 ms) : 0, 6228
Remote Config [baseline] (758.284 µs) : 0, 758
Remote Config [candidate] (756.494 µs) : 0, 756
Telemetry [baseline] (12.28 ms) : 0, 12280
Telemetry [candidate] (12.098 ms) : 0, 12098
section iast
BytebuddyAgent [baseline] (804.753 ms) : 0, 804753
BytebuddyAgent [candidate] (809.187 ms) : 0, 809187
GlobalTracer [baseline] (231.403 ms) : 0, 231403
GlobalTracer [candidate] (232.313 ms) : 0, 232313
IAST [baseline] (26.164 ms) : 0, 26164
IAST [candidate] (25.423 ms) : 0, 25423
AppSec [baseline] (56.085 ms) : 0, 56085
AppSec [candidate] (56.884 ms) : 0, 56884
Debugger [baseline] (5.985 ms) : 0, 5985
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (604.557 µs) : 0, 605
Remote Config [candidate] (597.818 µs) : 0, 598
Telemetry [baseline] (7.917 ms) : 0, 7917
Telemetry [candidate] (7.989 ms) : 0, 7989
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (810.149 ms) : 0, 810149
BytebuddyAgent [candidate] (804.591 ms) : 0, 804591
GlobalTracer [baseline] (233.104 ms) : 0, 233104
GlobalTracer [candidate] (232.227 ms) : 0, 232227
IAST [baseline] (27.381 ms) : 0, 27381
IAST [candidate] (28.818 ms) : 0, 28818
AppSec [baseline] (55.119 ms) : 0, 55119
AppSec [candidate] (53.217 ms) : 0, 53217
Debugger [baseline] (6.019 ms) : 0, 6019
Debugger [candidate] (5.955 ms) : 0, 5955
Remote Config [baseline] (588.91 µs) : 0, 589
Remote Config [candidate] (589.856 µs) : 0, 590
Telemetry [baseline] (7.89 ms) : 0, 7890
Telemetry [candidate] (7.927 ms) : 0, 7927
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.151 ms) : 0, 800151
BytebuddyAgent [candidate] (803.419 ms) : 0, 803419
GlobalTracer [baseline] (231.198 ms) : 0, 231198
GlobalTracer [candidate] (232.164 ms) : 0, 232164
IAST [baseline] (29.324 ms) : 0, 29324
IAST [candidate] (29.483 ms) : 0, 29483
AppSec [baseline] (51.483 ms) : 0, 51483
AppSec [candidate] (51.446 ms) : 0, 51446
Debugger [baseline] (6.051 ms) : 0, 6051
Debugger [candidate] (6.038 ms) : 0, 6038
Remote Config [baseline] (593.296 µs) : 0, 593
Remote Config [candidate] (597.714 µs) : 0, 598
Telemetry [baseline] (7.814 ms) : 0, 7814
Telemetry [candidate] (7.812 ms) : 0, 7812

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.034 s) : 0, 1034087
Total [baseline] (10.597 s) : 0, 10597162
Agent [candidate] (1.027 s) : 0, 1026700
Total [candidate] (10.63 s) : 0, 10630389
section appsec
Agent [baseline] (1.191 s) : 0, 1191468
Total [baseline] (10.883 s) : 0, 10882764
Agent [candidate] (1.185 s) : 0, 1185109
Total [candidate] (10.758 s) : 0, 10758494
section iast
Agent [baseline] (1.157 s) : 0, 1157475
Total [baseline] (10.887 s) : 0, 10887092
Agent [candidate] (1.16 s) : 0, 1159666
Total [candidate] (10.937 s) : 0, 10937420
section profiling
Agent [baseline] (1.279 s) : 0, 1279394
Total [baseline] (10.954 s) : 0, 10954163
Agent [candidate] (1.271 s) : 0, 1270596
Total [candidate] (10.978 s) : 0, 10978201

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.034 s	-
Agent	appsec	1.191 s	157.381 ms (15.2%)
Agent	iast	1.157 s	123.389 ms (11.9%)
Agent	profiling	1.279 s	245.308 ms (23.7%)
Total	tracing	10.597 s	-
Total	appsec	10.883 s	285.603 ms (2.7%)
Total	iast	10.887 s	289.93 ms (2.7%)
Total	profiling	10.954 s	357.001 ms (3.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	appsec	1.185 s	158.408 ms (15.4%)
Agent	iast	1.16 s	132.966 ms (13.0%)
Agent	profiling	1.271 s	243.896 ms (23.8%)
Total	tracing	10.63 s	-
Total	appsec	10.758 s	128.106 ms (1.2%)
Total	iast	10.937 s	307.032 ms (2.9%)
Total	profiling	10.978 s	347.812 ms (3.3%)

gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.527 ms) : 0, 689527
BytebuddyAgent [candidate] (683.594 ms) : 0, 683594
GlobalTracer [baseline] (242.973 ms) : 0, 242973
GlobalTracer [candidate] (240.847 ms) : 0, 240847
AppSec [baseline] (58.246 ms) : 0, 58246
AppSec [candidate] (59.139 ms) : 0, 59139
Debugger [baseline] (7.646 ms) : 0, 7646
Debugger [candidate] (7.645 ms) : 0, 7645
Remote Config [baseline] (755.364 µs) : 0, 755
Remote Config [candidate] (751.087 µs) : 0, 751
Telemetry [baseline] (11.343 ms) : 0, 11343
Telemetry [candidate] (11.285 ms) : 0, 11285
section appsec
BytebuddyAgent [baseline] (712.95 ms) : 0, 712950
BytebuddyAgent [candidate] (712.084 ms) : 0, 712084
GlobalTracer [baseline] (237.95 ms) : 0, 237950
GlobalTracer [candidate] (234.987 ms) : 0, 234987
AppSec [baseline] (180.731 ms) : 0, 180731
AppSec [candidate] (179.153 ms) : 0, 179153
Debugger [baseline] (5.949 ms) : 0, 5949
Debugger [candidate] (5.789 ms) : 0, 5789
Remote Config [baseline] (621.77 µs) : 0, 622
Remote Config [candidate] (604.938 µs) : 0, 605
Telemetry [baseline] (7.319 ms) : 0, 7319
Telemetry [candidate] (7.182 ms) : 0, 7182
IAST [baseline] (22.212 ms) : 0, 22212
IAST [candidate] (21.704 ms) : 0, 21704
section iast
BytebuddyAgent [baseline] (805.685 ms) : 0, 805685
BytebuddyAgent [candidate] (807.008 ms) : 0, 807008
GlobalTracer [baseline] (231.372 ms) : 0, 231372
GlobalTracer [candidate] (232.679 ms) : 0, 232679
AppSec [baseline] (55.24 ms) : 0, 55240
AppSec [candidate] (53.95 ms) : 0, 53950
Debugger [baseline] (6.061 ms) : 0, 6061
Debugger [candidate] (6.047 ms) : 0, 6047
Remote Config [baseline] (594.002 µs) : 0, 594
Remote Config [candidate] (591.309 µs) : 0, 591
Telemetry [baseline] (7.99 ms) : 0, 7990
Telemetry [candidate] (7.935 ms) : 0, 7935
IAST [baseline] (26.953 ms) : 0, 26953
IAST [candidate] (27.777 ms) : 0, 27777
section profiling
ProfilingAgent [baseline] (107.979 ms) : 0, 107979
ProfilingAgent [candidate] (105.86 ms) : 0, 105860
BytebuddyAgent [baseline] (679.472 ms) : 0, 679472
BytebuddyAgent [candidate] (673.923 ms) : 0, 673923
GlobalTracer [baseline] (361.072 ms) : 0, 361072
GlobalTracer [candidate] (360.339 ms) : 0, 360339
AppSec [baseline] (64.598 ms) : 0, 64598
AppSec [candidate] (64.723 ms) : 0, 64723
Debugger [baseline] (6.197 ms) : 0, 6197
Debugger [candidate] (6.152 ms) : 0, 6152
Remote Config [baseline] (658.963 µs) : 0, 659
Remote Config [candidate] (648.918 µs) : 0, 649
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (8.24 ms) : 0, 8240
Profiling [baseline] (108.003 ms) : 0, 108003
Profiling [candidate] (105.886 ms) : 0, 105886

Load

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352
dateFormat X
axisFormat %s
section baseline
no_agent (386.833 µs) : 367, 407
. : milestone, 387,
iast (535.844 µs) : 512, 559
. : milestone, 536,
iast_FULL (755.737 µs) : 732, 780
. : milestone, 756,
iast_GLOBAL (585.09 µs) : 562, 608
. : milestone, 585,
iast_HARDCODED_SECRET_DISABLED (530.703 µs) : 507, 554
. : milestone, 531,
iast_INACTIVE (473.184 µs) : 451, 496
. : milestone, 473,
iast_TELEMETRY_OFF (521.501 µs) : 500, 543
. : milestone, 522,
tracing (469.46 µs) : 447, 492
. : milestone, 469,
section candidate
no_agent (390.173 µs) : 370, 411
. : milestone, 390,
iast (542.631 µs) : 519, 566
. : milestone, 543,
iast_FULL (754.693 µs) : 731, 778
. : milestone, 755,
iast_GLOBAL (582.01 µs) : 559, 605
. : milestone, 582,
iast_HARDCODED_SECRET_DISABLED (541.59 µs) : 518, 565
. : milestone, 542,
iast_INACTIVE (480.898 µs) : 458, 504
. : milestone, 481,
iast_TELEMETRY_OFF (525.941 µs) : 504, 548
. : milestone, 526,
tracing (465.445 µs) : 443, 487
. : milestone, 465,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	386.833 µs [366.554 µs, 407.111 µs]	-
iast	535.844 µs [512.458 µs, 559.229 µs]	149.011 µs (38.5%)
iast_FULL	755.737 µs [731.803 µs, 779.672 µs]	368.904 µs (95.4%)
iast_GLOBAL	585.09 µs [561.724 µs, 608.457 µs]	198.258 µs (51.3%)
iast_HARDCODED_SECRET_DISABLED	530.703 µs [507.453 µs, 553.954 µs]	143.87 µs (37.2%)
iast_INACTIVE	473.184 µs [450.565 µs, 495.803 µs]	86.351 µs (22.3%)
iast_TELEMETRY_OFF	521.501 µs [499.857 µs, 543.144 µs]	134.668 µs (34.8%)
tracing	469.46 µs [447.231 µs, 491.688 µs]	82.627 µs (21.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	390.173 µs [369.669 µs, 410.678 µs]	-
iast	542.631 µs [519.307 µs, 565.955 µs]	152.458 µs (39.1%)
iast_FULL	754.693 µs [731.002 µs, 778.385 µs]	364.52 µs (93.4%)
iast_GLOBAL	582.01 µs [558.842 µs, 605.177 µs]	191.836 µs (49.2%)
iast_HARDCODED_SECRET_DISABLED	541.59 µs [518.422 µs, 564.758 µs]	151.416 µs (38.8%)
iast_INACTIVE	480.898 µs [458.086 µs, 503.71 µs]	90.725 µs (23.3%)
iast_TELEMETRY_OFF	525.941 µs [504.065 µs, 547.817 µs]	135.768 µs (34.8%)
tracing	465.445 µs [443.479 µs, 487.411 µs]	75.271 µs (19.3%)

Dacapo

fix: Maven smoke test are timed out in CI
9efa172

bric3 added the tag: no release notes Changes to exclude from release notes label Jun 16, 2025

bric3 marked this pull request as ready for review June 16, 2025 15:23

bric3 requested a review from a team as a code owner June 16, 2025 15:23

bric3 requested a review from daniel-mohedano June 16, 2025 15:23

AlexeyKuznetsov-DD approved these changes Jun 16, 2025
View reviewed changes

sarahchen6 approved these changes Jun 16, 2025
View reviewed changes

smola approved these changes Jun 16, 2025
View reviewed changes

smola added the comp: ci visibility Continuous Integration Visibility label Jun 16, 2025

PerfectSlayer changed the title ~~fix: Maven smoke test are timed out in CI~~Fix Maven smoke test timeout in CIJun 16, 2025

daniel-mohedano approved these changes Jun 16, 2025
View reviewed changes

bric3 merged commit 4fa48aa into masterJun 16, 2025

bric3 deleted the bdu/increasemaven-smoke-test-timeout branch June 16, 2025 16:04

github-actionsBot added this to the 1.50.0 milestone Jun 16, 2025

bric3 mentioned this pull request Jun 16, 2025
Increase again Maven smoke tests timeout #8997
Merged

Navigation Menu

Uh oh!

Fix Maven smoke test timeout in CI - #8993

Fix Maven smoke test timeout in CI#8993
bric3 merged 1 commit into
masterfrom
bdu/increasemaven-smoke-test-timeout

bric3 commented Jun 16, 2025

Uh oh!

pr-commenterBot commented Jun 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

bric3 commented Jun 16, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Jun 16, 2025

Benchmarks

Startup

Parameters

Summary

Load

Dacapo

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants