Update system-tests to 344e4abc912ada8004f2d0828a09efb65b5ee1e1 by smola · Pull Request #8512 · DataDog/dd-trace-java

smola · 2025-03-05T08:17:09Z

What Does This Do

Includes:

[java] Fix flaky test on openliberty log check system-tests#4248

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-03-05T08:55:21Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/system-tests-update
git_commit_date	1741124606	1741162565
git_commit_sha	`7787c9d`	`2d7ddc0`
release_version	1.47.0-SNAPSHOT~7787c9ddf9	1.47.0-SNAPSHOT~2d7ddc0c26

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741164874	1741164874
ci_job_id	833100041	833100041
ci_pipeline_id	57752355	57752355
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-raycvyd1-project-304-concurrent-0-2wybpjad 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-raycvyd1-project-304-concurrent-0-2wybpjad 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 7 unstable metrics.

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039436
Total [baseline] (8.692 s) : 0, 8691560
Agent [candidate] (1.046 s) : 0, 1046363
Total [candidate] (8.683 s) : 0, 8682815
section iast
Agent [baseline] (1.168 s) : 0, 1167951
Total [baseline] (9.202 s) : 0, 9202063
Agent [candidate] (1.168 s) : 0, 1168034
Total [candidate] (9.222 s) : 0, 9221802
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.171 s) : 0, 1171359
Total [baseline] (9.225 s) : 0, 9224544
Agent [candidate] (1.173 s) : 0, 1172752
Total [candidate] (9.218 s) : 0, 9217562
section iast_TELEMETRY_OFF
Agent [baseline] (1.166 s) : 0, 1165890
Total [baseline] (9.25 s) : 0, 9250183
Agent [candidate] (1.166 s) : 0, 1166264
Total [candidate] (9.233 s) : 0, 9233491

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	iast	1.168 s	128.515 ms (12.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.171 s	131.923 ms (12.7%)
Agent	iast_TELEMETRY_OFF	1.166 s	126.454 ms (12.2%)
Total	tracing	8.692 s	-
Total	iast	9.202 s	510.503 ms (5.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.225 s	532.984 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.25 s	558.624 ms (6.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	iast	1.168 s	121.671 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.173 s	126.39 ms (12.1%)
Agent	iast_TELEMETRY_OFF	1.166 s	119.901 ms (11.5%)
Total	tracing	8.683 s	-
Total	iast	9.222 s	538.987 ms (6.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.218 s	534.746 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.233 s	550.676 ms (6.3%)

gantt
title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.62 ms) : 0, 717620
BytebuddyAgent [candidate] (722.834 ms) : 0, 722834
GlobalTracer [baseline] (239.825 ms) : 0, 239825
GlobalTracer [candidate] (241.083 ms) : 0, 241083
AppSec [baseline] (55.634 ms) : 0, 55634
AppSec [candidate] (55.898 ms) : 0, 55898
Remote Config [baseline] (689.592 µs) : 0, 690
Remote Config [candidate] (700.516 µs) : 0, 701
Telemetry [baseline] (10.617 ms) : 0, 10617
Telemetry [candidate] (10.844 ms) : 0, 10844
section iast
BytebuddyAgent [baseline] (834.22 ms) : 0, 834220
BytebuddyAgent [candidate] (834.812 ms) : 0, 834812
GlobalTracer [baseline] (230.26 ms) : 0, 230260
GlobalTracer [candidate] (229.896 ms) : 0, 229896
IAST [baseline] (22.604 ms) : 0, 22604
IAST [candidate] (22.69 ms) : 0, 22690
AppSec [baseline] (56.723 ms) : 0, 56723
AppSec [candidate] (56.519 ms) : 0, 56519
Remote Config [baseline] (604.88 µs) : 0, 605
Remote Config [candidate] (604.784 µs) : 0, 605
Telemetry [baseline] (8.59 ms) : 0, 8590
Telemetry [candidate] (8.579 ms) : 0, 8579
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (835.725 ms) : 0, 835725
BytebuddyAgent [candidate] (837.642 ms) : 0, 837642
GlobalTracer [baseline] (230.741 ms) : 0, 230741
GlobalTracer [candidate] (230.653 ms) : 0, 230653
IAST [baseline] (23.706 ms) : 0, 23706
IAST [candidate] (22.967 ms) : 0, 22967
AppSec [baseline] (56.731 ms) : 0, 56731
AppSec [candidate] (57.14 ms) : 0, 57140
Remote Config [baseline] (609.54 µs) : 0, 610
Remote Config [candidate] (620.13 µs) : 0, 620
Telemetry [baseline] (8.841 ms) : 0, 8841
Telemetry [candidate] (8.75 ms) : 0, 8750
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (832.301 ms) : 0, 832301
BytebuddyAgent [candidate] (832.453 ms) : 0, 832453
GlobalTracer [baseline] (230.536 ms) : 0, 230536
GlobalTracer [candidate] (230.463 ms) : 0, 230463
IAST [baseline] (25.267 ms) : 0, 25267
IAST [candidate] (24.615 ms) : 0, 24615
AppSec [baseline] (53.691 ms) : 0, 53691
AppSec [candidate] (54.447 ms) : 0, 54447
Remote Config [baseline] (609.4 µs) : 0, 609
Remote Config [candidate] (612.576 µs) : 0, 613
Telemetry [baseline] (8.504 ms) : 0, 8504
Telemetry [candidate] (8.647 ms) : 0, 8647

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1041773
Total [baseline] (10.414 s) : 0, 10413735
Agent [candidate] (1.048 s) : 0, 1048365
Total [candidate] (10.452 s) : 0, 10452039
section appsec
Agent [baseline] (1.183 s) : 0, 1182861
Total [baseline] (10.726 s) : 0, 10725561
Agent [candidate] (1.183 s) : 0, 1183104
Total [candidate] (10.736 s) : 0, 10736098
section iast
Agent [baseline] (1.178 s) : 0, 1177592
Total [baseline] (10.918 s) : 0, 10917839
Agent [candidate] (1.172 s) : 0, 1172198
Total [candidate] (10.974 s) : 0, 10974238
section profiling
Agent [baseline] (1.27 s) : 0, 1270069
Total [baseline] (10.807 s) : 0, 10806514
Agent [candidate] (1.267 s) : 0, 1266768
Total [candidate] (10.867 s) : 0, 10866794

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	appsec	1.183 s	141.088 ms (13.5%)
Agent	iast	1.178 s	135.819 ms (13.0%)
Agent	profiling	1.27 s	228.296 ms (21.9%)
Total	tracing	10.414 s	-
Total	appsec	10.726 s	311.826 ms (3.0%)
Total	iast	10.918 s	504.104 ms (4.8%)
Total	profiling	10.807 s	392.778 ms (3.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.183 s	134.739 ms (12.9%)
Agent	iast	1.172 s	123.833 ms (11.8%)
Agent	profiling	1.267 s	218.403 ms (20.8%)
Total	tracing	10.452 s	-
Total	appsec	10.736 s	284.059 ms (2.7%)
Total	iast	10.974 s	522.198 ms (5.0%)
Total	profiling	10.867 s	414.755 ms (4.0%)

gantt
title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.016 ms) : 0, 718016
BytebuddyAgent [candidate] (722.194 ms) : 0, 722194
GlobalTracer [baseline] (239.89 ms) : 0, 239890
GlobalTracer [candidate] (240.835 ms) : 0, 240835
AppSec [baseline] (55.962 ms) : 0, 55962
AppSec [candidate] (55.993 ms) : 0, 55993
Remote Config [baseline] (693.975 µs) : 0, 694
Remote Config [candidate] (696.411 µs) : 0, 696
Telemetry [baseline] (12.204 ms) : 0, 12204
Telemetry [candidate] (13.558 ms) : 0, 13558
section appsec
BytebuddyAgent [baseline] (734.401 ms) : 0, 734401
BytebuddyAgent [candidate] (735.334 ms) : 0, 735334
GlobalTracer [baseline] (236.666 ms) : 0, 236666
GlobalTracer [candidate] (236.087 ms) : 0, 236087
IAST [baseline] (21.446 ms) : 0, 21446
IAST [candidate] (21.443 ms) : 0, 21443
AppSec [baseline] (177.199 ms) : 0, 177199
AppSec [candidate] (177.102 ms) : 0, 177102
Remote Config [baseline] (666.849 µs) : 0, 667
Remote Config [candidate] (657.378 µs) : 0, 657
Telemetry [baseline] (8.279 ms) : 0, 8279
Telemetry [candidate] (8.272 ms) : 0, 8272
section iast
BytebuddyAgent [baseline] (841.656 ms) : 0, 841656
BytebuddyAgent [candidate] (837.638 ms) : 0, 837638
GlobalTracer [baseline] (231.324 ms) : 0, 231324
GlobalTracer [candidate] (230.436 ms) : 0, 230436
IAST [baseline] (23.685 ms) : 0, 23685
IAST [candidate] (22.821 ms) : 0, 22821
AppSec [baseline] (56.548 ms) : 0, 56548
AppSec [candidate] (57.102 ms) : 0, 57102
Remote Config [baseline] (607.08 µs) : 0, 607
Remote Config [candidate] (601.702 µs) : 0, 602
Telemetry [baseline] (8.706 ms) : 0, 8706
Telemetry [candidate] (8.677 ms) : 0, 8677
section profiling
BytebuddyAgent [baseline] (714.819 ms) : 0, 714819
BytebuddyAgent [candidate] (713.024 ms) : 0, 713024
GlobalTracer [baseline] (351.821 ms) : 0, 351821
GlobalTracer [candidate] (352.238 ms) : 0, 352238
AppSec [baseline] (55.9 ms) : 0, 55900
AppSec [candidate] (55.041 ms) : 0, 55041
Remote Config [baseline] (676.621 µs) : 0, 677
Remote Config [candidate] (666.578 µs) : 0, 667
Telemetry [baseline] (9.029 ms) : 0, 9029
Telemetry [candidate] (9.136 ms) : 0, 9136
ProfilingAgent [baseline] (97.119 ms) : 0, 97119
ProfilingAgent [candidate] (96.236 ms) : 0, 96236
Profiling [baseline] (97.144 ms) : 0, 97144
Profiling [candidate] (96.26 ms) : 0, 96260

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-05T08:24:53	2025-03-05T08:32:35
git_branch	master	smola/system-tests-update
git_commit_date	1741124606	1741162565
git_commit_sha	`7787c9d`	`2d7ddc0`
release_version	1.47.0-SNAPSHOT~7787c9ddf9	1.47.0-SNAPSHOT~2d7ddc0c26
start_time	2025-03-05T08:24:39	2025-03-05T08:32:21

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741163951	1741163951
ci_job_id	833100042	833100042
ci_pipeline_id	57752355	57752355
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-raycvyd1-project-304-concurrent-1-owxm605g 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-raycvyd1-project-304-concurrent-1-owxm605g 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section baseline
no_agent (385.484 µs) : 366, 405
. : milestone, 385,
iast (512.909 µs) : 491, 535
. : milestone, 513,
iast_FULL (729.277 µs) : 708, 751
. : milestone, 729,
iast_GLOBAL (566.561 µs) : 544, 590
. : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (514.026 µs) : 491, 537
. : milestone, 514,
iast_INACTIVE (457.344 µs) : 436, 479
. : milestone, 457,
iast_TELEMETRY_OFF (494.372 µs) : 472, 517
. : milestone, 494,
tracing (457.815 µs) : 437, 479
. : milestone, 458,
section candidate
no_agent (380.205 µs) : 358, 403
. : milestone, 380,
iast (512.01 µs) : 490, 534
. : milestone, 512,
iast_FULL (727.721 µs) : 706, 750
. : milestone, 728,
iast_GLOBAL (549.437 µs) : 528, 571
. : milestone, 549,
iast_HARDCODED_SECRET_DISABLED (504.185 µs) : 483, 526
. : milestone, 504,
iast_INACTIVE (459.292 µs) : 438, 480
. : milestone, 459,
iast_TELEMETRY_OFF (496.093 µs) : 474, 518
. : milestone, 496,
tracing (460.031 µs) : 439, 482
. : milestone, 460,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	385.484 µs [365.564 µs, 405.404 µs]	-
iast	512.909 µs [490.873 µs, 534.945 µs]	127.425 µs (33.1%)
iast_FULL	729.277 µs [707.527 µs, 751.027 µs]	343.793 µs (89.2%)
iast_GLOBAL	566.561 µs [543.532 µs, 589.591 µs]	181.077 µs (47.0%)
iast_HARDCODED_SECRET_DISABLED	514.026 µs [491.424 µs, 536.627 µs]	128.542 µs (33.3%)
iast_INACTIVE	457.344 µs [436.02 µs, 478.669 µs]	71.86 µs (18.6%)
iast_TELEMETRY_OFF	494.372 µs [472.052 µs, 516.693 µs]	108.888 µs (28.2%)
tracing	457.815 µs [436.983 µs, 478.646 µs]	72.331 µs (18.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	380.205 µs [357.895 µs, 402.514 µs]	-
iast	512.01 µs [490.061 µs, 533.959 µs]	131.805 µs (34.7%)
iast_FULL	727.721 µs [705.79 µs, 749.652 µs]	347.516 µs (91.4%)
iast_GLOBAL	549.437 µs [528.133 µs, 570.74 µs]	169.232 µs (44.5%)
iast_HARDCODED_SECRET_DISABLED	504.185 µs [482.573 µs, 525.797 µs]	123.981 µs (32.6%)
iast_INACTIVE	459.292 µs [438.212 µs, 480.372 µs]	79.088 µs (20.8%)
iast_TELEMETRY_OFF	496.093 µs [473.739 µs, 518.446 µs]	115.888 µs (30.5%)
tracing	460.031 µs [438.545 µs, 481.517 µs]	79.827 µs (21.0%)

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section baseline
no_agent (1.36 ms) : 1340, 1379
. : milestone, 1360,
appsec (1.715 ms) : 1691, 1739
. : milestone, 1715,
appsec_no_iast (1.733 ms) : 1709, 1756
. : milestone, 1733,
code_origins (1.667 ms) : 1634, 1701
. : milestone, 1667,
iast (1.521 ms) : 1496, 1546
. : milestone, 1521,
profiling (1.493 ms) : 1469, 1516
. : milestone, 1493,
tracing (1.473 ms) : 1447, 1498
. : milestone, 1473,
section candidate
no_agent (1.345 ms) : 1325, 1365
. : milestone, 1345,
appsec (1.712 ms) : 1688, 1736
. : milestone, 1712,
appsec_no_iast (1.735 ms) : 1711, 1759
. : milestone, 1735,
code_origins (1.666 ms) : 1633, 1699
. : milestone, 1666,
iast (1.513 ms) : 1489, 1537
. : milestone, 1513,
profiling (1.515 ms) : 1491, 1540
. : milestone, 1515,
tracing (1.485 ms) : 1461, 1509
. : milestone, 1485,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.36 ms [1.34 ms, 1.379 ms]	-
appsec	1.715 ms [1.691 ms, 1.739 ms]	355.012 µs (26.1%)
appsec_no_iast	1.733 ms [1.709 ms, 1.756 ms]	372.6 µs (27.4%)
code_origins	1.667 ms [1.634 ms, 1.701 ms]	307.508 µs (22.6%)
iast	1.521 ms [1.496 ms, 1.546 ms]	160.696 µs (11.8%)
profiling	1.493 ms [1.469 ms, 1.516 ms]	132.899 µs (9.8%)
tracing	1.473 ms [1.447 ms, 1.498 ms]	112.617 µs (8.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.345 ms [1.325 ms, 1.365 ms]	-
appsec	1.712 ms [1.688 ms, 1.736 ms]	366.748 µs (27.3%)
appsec_no_iast	1.735 ms [1.711 ms, 1.759 ms]	390.112 µs (29.0%)
code_origins	1.666 ms [1.633 ms, 1.699 ms]	320.708 µs (23.8%)
iast	1.513 ms [1.489 ms, 1.537 ms]	167.9 µs (12.5%)
profiling	1.515 ms [1.491 ms, 1.54 ms]	170.135 µs (12.6%)
tracing	1.485 ms [1.461 ms, 1.509 ms]	139.845 µs (10.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/system-tests-update
git_commit_date	1741124606	1741162565
git_commit_sha	`7787c9d`	`2d7ddc0`
release_version	1.47.0-SNAPSHOT~7787c9ddf9	1.47.0-SNAPSHOT~2d7ddc0c26

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1741164423	1741164423
ci_job_id	833100043	833100043
ci_pipeline_id	57752355	57752355
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-raycvyd1-project-304-concurrent-2-99vyqfuq 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-raycvyd1-project-304-concurrent-2-99vyqfuq 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section baseline
no_agent (1.485 ms) : 1473, 1496
. : milestone, 1485,
appsec (2.363 ms) : 2319, 2407
. : milestone, 2363,
iast (2.133 ms) : 2077, 2188
. : milestone, 2133,
iast_GLOBAL (2.17 ms) : 2115, 2225
. : milestone, 2170,
profiling (1.987 ms) : 1943, 2030
. : milestone, 1987,
tracing (1.957 ms) : 1914, 1999
. : milestone, 1957,
section candidate
no_agent (1.482 ms) : 1471, 1494
. : milestone, 1482,
appsec (2.363 ms) : 2319, 2407
. : milestone, 2363,
iast (2.131 ms) : 2075, 2186
. : milestone, 2131,
iast_GLOBAL (2.174 ms) : 2119, 2230
. : milestone, 2174,
profiling (1.975 ms) : 1932, 2019
. : milestone, 1975,
tracing (1.963 ms) : 1921, 2006
. : milestone, 1963,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.485 ms [1.473 ms, 1.496 ms]	-
appsec	2.363 ms [2.319 ms, 2.407 ms]	878.268 µs (59.2%)
iast	2.133 ms [2.077 ms, 2.188 ms]	648.089 µs (43.7%)
iast_GLOBAL	2.17 ms [2.115 ms, 2.225 ms]	685.275 µs (46.2%)
profiling	1.987 ms [1.943 ms, 2.03 ms]	501.863 µs (33.8%)
tracing	1.957 ms [1.914 ms, 1.999 ms]	472.018 µs (31.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.482 ms [1.471 ms, 1.494 ms]	-
appsec	2.363 ms [2.319 ms, 2.407 ms]	880.973 µs (59.4%)
iast	2.131 ms [2.075 ms, 2.186 ms]	648.432 µs (43.7%)
iast_GLOBAL	2.174 ms [2.119 ms, 2.23 ms]	692.215 µs (46.7%)
profiling	1.975 ms [1.932 ms, 2.019 ms]	493.001 µs (33.3%)
tracing	1.963 ms [1.921 ms, 2.006 ms]	481.13 µs (32.5%)

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~2d7ddc0c26, baseline=1.47.0-SNAPSHOT~7787c9ddf9
dateFormat X
axisFormat %s
section baseline
no_agent (15.561 s) : 15561000, 15561000
. : milestone, 15561000,
appsec (15.133 s) : 15133000, 15133000
. : milestone, 15133000,
iast (18.938 s) : 18938000, 18938000
. : milestone, 18938000,
iast_GLOBAL (18.07 s) : 18070000, 18070000
. : milestone, 18070000,
profiling (15.069 s) : 15069000, 15069000
. : milestone, 15069000,
tracing (15.0 s) : 15000000, 15000000
. : milestone, 15000000,
section candidate
no_agent (15.228 s) : 15228000, 15228000
. : milestone, 15228000,
appsec (15.24 s) : 15240000, 15240000
. : milestone, 15240000,
iast (18.615 s) : 18615000, 18615000
. : milestone, 18615000,
iast_GLOBAL (17.982 s) : 17982000, 17982000
. : milestone, 17982000,
profiling (15.165 s) : 15165000, 15165000
. : milestone, 15165000,
tracing (14.781 s) : 14781000, 14781000
. : milestone, 14781000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.561 s [15.561 s, 15.561 s]	-
appsec	15.133 s [15.133 s, 15.133 s]	-428.0 ms (-2.8%)
iast	18.938 s [18.938 s, 18.938 s]	3.377 s (21.7%)
iast_GLOBAL	18.07 s [18.07 s, 18.07 s]	2.509 s (16.1%)
profiling	15.069 s [15.069 s, 15.069 s]	-492.0 ms (-3.2%)
tracing	15.0 s [15.0 s, 15.0 s]	-561.0 ms (-3.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.228 s [15.228 s, 15.228 s]	-
appsec	15.24 s [15.24 s, 15.24 s]	12.0 ms (0.1%)
iast	18.615 s [18.615 s, 18.615 s]	3.387 s (22.2%)
iast_GLOBAL	17.982 s [17.982 s, 17.982 s]	2.754 s (18.1%)
profiling	15.165 s [15.165 s, 15.165 s]	-63.0 ms (-0.4%)
tracing	14.781 s [14.781 s, 14.781 s]	-447.0 ms (-2.9%)

PerfectSlayer

Thanks for updating the tests!

Update system-tests to 344e4abc912ada8004f2d0828a09efb65b5ee1e1
2d7ddc0

smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 5, 2025

smola marked this pull request as ready for review March 5, 2025 08:57

smola requested a review from a team as a code owner March 5, 2025 08:57

smola requested a review from amarziali March 5, 2025 08:57

smola enabled auto-merge (squash) March 5, 2025 09:31

PerfectSlayer approved these changes Mar 5, 2025
View reviewed changes

smola merged commit 1836b11 into masterMar 5, 2025

smola deleted the smola/system-tests-update branch March 5, 2025 10:34

github-actionsBot added this to the 1.47.0 milestone Mar 5, 2025

Navigation Menu

Uh oh!

Update system-tests to 344e4abc912ada8004f2d0828a09efb65b5ee1e1 - #8512

Update system-tests to 344e4abc912ada8004f2d0828a09efb65b5ee1e1#8512
smola merged 1 commit into
masterfrom
smola/system-tests-update

smola commented Mar 5, 2025

Uh oh!

pr-commenterBot commented Mar 5, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

smola commented Mar 5, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Mar 5, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants