Nikita tkachenko/test env 08e0fa3c435603d3b69ac98b0120f4507a356d0a by nikita-tkachenko-datadog · Pull Request #8498 · DataDog/dd-trace-java

nikita-tkachenko-datadog · 2025-03-04T11:57:36Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-03-04T12:34:50Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-env-08e0fa3c435603d3b69ac98b0120f4507a356d0a
git_commit_date	1741082792	1740744591
git_commit_sha	`8f945b0`	`08e0fa3`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~08e0fa3c43

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741091640	1741091640
ci_job_id	831517656	831517656
ci_pipeline_id	57626459	57626459
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-xk9phren-project-304-concurrent-0-tv0v8hut 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-xk9phren-project-304-concurrent-0-tv0v8hut 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 21 performance improvements and 6 performance regressions! Performance is the same for 28 metrics, 8 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent	better [-780.555ms; -528.724ms] or [-7.834%; -5.307%]	9.309s	9.963s
scenario:startup:insecure-bank:iast:Agent.start	better [-98.787ms; -81.678ms] or [-7.828%; -6.472%]	1.172s	1.262s
scenario:startup:insecure-bank:iast:Remote Config	worse [+88.183µs; +138.955µs] or [+17.062%; +26.886%]	630.399µs	516.830µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent	better [-754.003ms; -563.870ms] or [-7.626%; -5.703%]	9.229s	9.888s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start	better [-95.130ms; -78.005ms] or [-7.554%; -6.194%]	1.173s	1.259s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent	better [-663.030ms; -483.351ms] or [-6.715%; -4.895%]	9.301s	9.874s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start	better [-84.884ms; -79.198ms] or [-6.796%; -6.341%]	1.167s	1.249s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config	worse [+80.262µs; +110.137µs] or [+15.558%; +21.349%]	611.086µs	515.886µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry	better [-3.510ms; -3.242ms] or [-29.196%; -26.961%]	8.647ms	12.023ms
scenario:startup:insecure-bank:tracing:Agent	better [-679.340ms; -511.162ms] or [-7.307%; -5.498%]	8.702s	9.297s
scenario:startup:insecure-bank:tracing:Agent.start	better [-102.560ms; -64.603ms] or [-9.035%; -5.691%]	1.052s	1.135s
scenario:startup:insecure-bank:tracing:Remote Config	worse [+106.763µs; +141.341µs] or [+18.182%; +24.071%]	711.229µs	587.178µs
scenario:startup:petclinic:appsec:Agent	better [-974.981ms; -807.068ms] or [-8.343%; -6.906%]	10.796s	11.687s
scenario:startup:petclinic:appsec:Agent.start	better [-96.952ms; -80.390ms] or [-7.619%; -6.318%]	1.184s	1.272s
scenario:startup:petclinic:appsec:Telemetry	better [-4.315ms; -3.890ms] or [-34.799%; -31.367%]	8.298ms	12.401ms
scenario:startup:petclinic:iast:Agent	better [-1048.927ms; -735.263ms] or [-8.770%; -6.148%]	11.068s	11.960s
scenario:startup:petclinic:iast:Agent.start	better [-96.561ms; -77.773ms] or [-7.657%; -6.167%]	1.174s	1.261s
scenario:startup:petclinic:iast:Remote Config	worse [+72.552µs; +111.128µs] or [+13.721%; +21.017%]	620.599µs	528.760µs
scenario:startup:petclinic:profiling:Agent	better [-983.434ms; -725.074ms] or [-8.357%; -6.161%]	10.914s	11.768s
scenario:startup:petclinic:profiling:Agent.start	better [-124.697ms; -115.631ms] or [-9.032%; -8.375%]	1.261s	1.381s
scenario:startup:petclinic:profiling:ProfilingAgent	better [-28.371ms; -26.136ms] or [-23.015%; -21.201%]	96.020ms	123.273ms
scenario:startup:petclinic:profiling:Remote Config	worse [+40.035µs; +93.796µs] or [+6.546%; +15.335%]	678.552µs	611.636µs
scenario:startup:petclinic:profiling:Telemetry	better [-3.703ms; -3.215ms] or [-29.490%; -25.610%]	9.096ms	12.555ms
scenario:startup:petclinic:profiling:Profiling	better [-28.366ms; -26.131ms] or [-23.007%; -21.194%]	96.045ms	123.294ms
scenario:startup:petclinic:tracing:Agent	better [-1.251s; -0.902s] or [-10.820%; -7.804%]	10.485s	11.561s
scenario:startup:petclinic:tracing:Agent.start	better [-107.373ms; -77.413ms] or [-9.445%; -6.810%]	1.044s	1.137s
scenario:startup:petclinic:tracing:Remote Config	worse [+77.018µs; +133.120µs] or [+12.890%; +22.279%]	702.574µs	597.505µs

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.135 s) : 0, 1135110
Total [baseline] (9.297 s) : 0, 9296924
Agent [candidate] (1.052 s) : 0, 1051528
Total [candidate] (8.702 s) : 0, 8701673
section iast
Agent [baseline] (1.262 s) : 0, 1262028
Total [baseline] (9.963 s) : 0, 9963372
Agent [candidate] (1.172 s) : 0, 1171796
Total [candidate] (9.309 s) : 0, 9308732
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.259 s) : 0, 1259380
Total [baseline] (9.888 s) : 0, 9887813
Agent [candidate] (1.173 s) : 0, 1172812
Total [candidate] (9.229 s) : 0, 9228877
section iast_TELEMETRY_OFF
Agent [baseline] (1.249 s) : 0, 1249067
Total [baseline] (9.874 s) : 0, 9874045
Agent [candidate] (1.167 s) : 0, 1167026
Total [candidate] (9.301 s) : 0, 9300855

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.135 s	-
Agent	iast	1.262 s	126.918 ms (11.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.259 s	124.27 ms (10.9%)
Agent	iast_TELEMETRY_OFF	1.249 s	113.957 ms (10.0%)
Total	tracing	9.297 s	-
Total	iast	9.963 s	666.447 ms (7.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.888 s	590.889 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.874 s	577.121 ms (6.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.172 s	120.268 ms (11.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.173 s	121.284 ms (11.5%)
Agent	iast_TELEMETRY_OFF	1.167 s	115.498 ms (11.0%)
Total	tracing	8.702 s	-
Total	iast	9.309 s	607.059 ms (7.0%)
Total	iast_HARDCODED_SECRET_DISABLED	9.229 s	527.204 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.301 s	599.181 ms (6.9%)

gantt
title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.007 ms) : 0, 717007
BytebuddyAgent [candidate] (725.542 ms) : 0, 725542
GlobalTracer [baseline] (239.062 ms) : 0, 239062
GlobalTracer [candidate] (241.2 ms) : 0, 241200
AppSec [baseline] (55.46 ms) : 0, 55460
AppSec [candidate] (55.921 ms) : 0, 55921
Remote Config [baseline] (587.178 µs) : 0, 587
Remote Config [candidate] (711.229 µs) : 0, 711
Telemetry [baseline] (12.467 ms) : 0, 12467
Telemetry [candidate] (13.052 ms) : 0, 13052
Debugger [baseline] (95.529 ms) : 0, 95529
section iast
BytebuddyAgent [baseline] (837.01 ms) : 0, 837010
BytebuddyAgent [candidate] (835.411 ms) : 0, 835411
GlobalTracer [baseline] (231.022 ms) : 0, 231022
GlobalTracer [candidate] (231.054 ms) : 0, 231054
IAST [baseline] (22.698 ms) : 0, 22698
IAST [candidate] (23.094 ms) : 0, 23094
AppSec [baseline] (57.31 ms) : 0, 57310
AppSec [candidate] (57.711 ms) : 0, 57711
Remote Config [baseline] (516.83 µs) : 0, 517
Remote Config [candidate] (630.399 µs) : 0, 630
Telemetry [baseline] (14.258 ms) : 0, 14258
Telemetry [candidate] (8.909 ms) : 0, 8909
Debugger [baseline] (84.164 ms) : 0, 84164
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (835.175 ms) : 0, 835175
BytebuddyAgent [candidate] (836.123 ms) : 0, 836123
GlobalTracer [baseline] (230.648 ms) : 0, 230648
GlobalTracer [candidate] (231.152 ms) : 0, 231152
IAST [baseline] (22.706 ms) : 0, 22706
IAST [candidate] (23.093 ms) : 0, 23093
AppSec [baseline] (57.404 ms) : 0, 57404
AppSec [candidate] (57.907 ms) : 0, 57907
Remote Config [baseline] (529.409 µs) : 0, 529
Remote Config [candidate] (637.721 µs) : 0, 638
Telemetry [baseline] (13.637 ms) : 0, 13637
Telemetry [candidate] (8.903 ms) : 0, 8903
Debugger [baseline] (84.284 ms) : 0, 84284
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (829.887 ms) : 0, 829887
BytebuddyAgent [candidate] (832.923 ms) : 0, 832923
GlobalTracer [baseline] (229.567 ms) : 0, 229567
GlobalTracer [candidate] (230.247 ms) : 0, 230247
IAST [baseline] (23.736 ms) : 0, 23736
IAST [candidate] (24.822 ms) : 0, 24822
AppSec [baseline] (55.201 ms) : 0, 55201
AppSec [candidate] (54.817 ms) : 0, 54817
Remote Config [baseline] (515.886 µs) : 0, 516
Remote Config [candidate] (611.086 µs) : 0, 611
Telemetry [baseline] (12.023 ms) : 0, 12023
Telemetry [candidate] (8.647 ms) : 0, 8647
Debugger [baseline] (83.187 ms) : 0, 83187

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.137 s) : 0, 1136809
Total [baseline] (11.561 s) : 0, 11561255
Agent [candidate] (1.044 s) : 0, 1044416
Total [candidate] (10.485 s) : 0, 10484665
section appsec
Agent [baseline] (1.272 s) : 0, 1272422
Total [baseline] (11.687 s) : 0, 11686899
Agent [candidate] (1.184 s) : 0, 1183751
Total [candidate] (10.796 s) : 0, 10795875
section iast
Agent [baseline] (1.261 s) : 0, 1261108
Total [baseline] (11.96 s) : 0, 11959933
Agent [candidate] (1.174 s) : 0, 1173941
Total [candidate] (11.068 s) : 0, 11067838
section profiling
Agent [baseline] (1.381 s) : 0, 1380666
Total [baseline] (11.768 s) : 0, 11767870
Agent [candidate] (1.261 s) : 0, 1260503
Total [candidate] (10.914 s) : 0, 10913616

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.137 s	-
Agent	appsec	1.272 s	135.613 ms (11.9%)
Agent	iast	1.261 s	124.299 ms (10.9%)
Agent	profiling	1.381 s	243.858 ms (21.5%)
Total	tracing	11.561 s	-
Total	appsec	11.687 s	125.644 ms (1.1%)
Total	iast	11.96 s	398.678 ms (3.4%)
Total	profiling	11.768 s	206.615 ms (1.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	appsec	1.184 s	139.335 ms (13.3%)
Agent	iast	1.174 s	129.525 ms (12.4%)
Agent	profiling	1.261 s	216.087 ms (20.7%)
Total	tracing	10.485 s	-
Total	appsec	10.796 s	311.21 ms (3.0%)
Total	iast	11.068 s	583.172 ms (5.6%)
Total	profiling	10.914 s	428.951 ms (4.1%)

gantt
title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.594 ms) : 0, 716594
BytebuddyAgent [candidate] (721.205 ms) : 0, 721205
GlobalTracer [baseline] (239.755 ms) : 0, 239755
GlobalTracer [candidate] (240.806 ms) : 0, 240806
AppSec [baseline] (55.833 ms) : 0, 55833
AppSec [candidate] (55.959 ms) : 0, 55959
Remote Config [baseline] (597.505 µs) : 0, 598
Remote Config [candidate] (702.574 µs) : 0, 703
Telemetry [baseline] (12.419 ms) : 0, 12419
Telemetry [candidate] (10.709 ms) : 0, 10709
Debugger [baseline] (96.624 ms) : 0, 96624
section appsec
BytebuddyAgent [baseline] (734.651 ms) : 0, 734651
BytebuddyAgent [candidate] (735.394 ms) : 0, 735394
GlobalTracer [baseline] (236.696 ms) : 0, 236696
GlobalTracer [candidate] (236.118 ms) : 0, 236118
AppSec [baseline] (177.735 ms) : 0, 177735
AppSec [candidate] (177.493 ms) : 0, 177493
Remote Config [baseline] (639.85 µs) : 0, 640
Remote Config [candidate] (661.974 µs) : 0, 662
Telemetry [baseline] (12.401 ms) : 0, 12401
Telemetry [candidate] (8.298 ms) : 0, 8298
Debugger [baseline] (84.553 ms) : 0, 84553
IAST [baseline] (21.442 ms) : 0, 21442
IAST [candidate] (21.621 ms) : 0, 21621
section iast
BytebuddyAgent [baseline] (835.819 ms) : 0, 835819
BytebuddyAgent [candidate] (839.16 ms) : 0, 839160
GlobalTracer [baseline] (230.749 ms) : 0, 230749
GlobalTracer [candidate] (229.829 ms) : 0, 229829
AppSec [baseline] (57.157 ms) : 0, 57157
AppSec [candidate] (57.621 ms) : 0, 57621
Remote Config [baseline] (528.76 µs) : 0, 529
Remote Config [candidate] (620.599 µs) : 0, 621
Telemetry [baseline] (15.288 ms) : 0, 15288
Telemetry [candidate] (8.743 ms) : 0, 8743
Debugger [baseline] (83.6 ms) : 0, 83600
IAST [baseline] (22.96 ms) : 0, 22960
IAST [candidate] (22.911 ms) : 0, 22911
section profiling
ProfilingAgent [baseline] (123.273 ms) : 0, 123273
ProfilingAgent [candidate] (96.02 ms) : 0, 96020
BytebuddyAgent [baseline] (709.119 ms) : 0, 709119
BytebuddyAgent [candidate] (708.791 ms) : 0, 708791
GlobalTracer [baseline] (350.947 ms) : 0, 350947
GlobalTracer [candidate] (350.991 ms) : 0, 350991
AppSec [baseline] (55.214 ms) : 0, 55214
AppSec [candidate] (54.609 ms) : 0, 54609
Remote Config [baseline] (611.636 µs) : 0, 612
Remote Config [candidate] (678.552 µs) : 0, 679
Telemetry [baseline] (12.555 ms) : 0, 12555
Telemetry [candidate] (9.096 ms) : 0, 9096
Profiling [baseline] (123.294 ms) : 0, 123294
Profiling [candidate] (96.045 ms) : 0, 96045
Debugger [baseline] (88.528 ms) : 0, 88528

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-04T12:03:03	2025-03-04T12:10:56
git_branch	master	nikita-tkachenko/test-env-08e0fa3c435603d3b69ac98b0120f4507a356d0a
git_commit_date	1741082792	1740744591
git_commit_sha	`8f945b0`	`08e0fa3`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~08e0fa3c43
start_time	2025-03-04T12:02:49	2025-03-04T12:10:42

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741090656	1741090656
ci_job_id	831517657	831517657
ci_pipeline_id	57626459	57626459
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-bniqe5xm-project-304-concurrent-2-qqvxr5q0 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-bniqe5xm-project-304-concurrent-2-qqvxr5q0 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (1.365 ms) : 1345, 1385
. : milestone, 1365,
appsec (1.725 ms) : 1701, 1749
. : milestone, 1725,
appsec_no_iast (1.727 ms) : 1703, 1751
. : milestone, 1727,
code_origins (1.679 ms) : 1647, 1711
. : milestone, 1679,
iast (1.511 ms) : 1487, 1536
. : milestone, 1511,
profiling (1.556 ms) : 1530, 1582
. : milestone, 1556,
tracing (1.476 ms) : 1452, 1499
. : milestone, 1476,
section candidate
no_agent (1.36 ms) : 1340, 1380
. : milestone, 1360,
appsec (1.738 ms) : 1715, 1761
. : milestone, 1738,
appsec_no_iast (1.73 ms) : 1705, 1755
. : milestone, 1730,
code_origins (1.704 ms) : 1671, 1736
. : milestone, 1704,
iast (1.514 ms) : 1491, 1538
. : milestone, 1514,
profiling (1.564 ms) : 1540, 1588
. : milestone, 1564,
tracing (1.521 ms) : 1494, 1547
. : milestone, 1521,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.365 ms [1.345 ms, 1.385 ms]	-
appsec	1.725 ms [1.701 ms, 1.749 ms]	359.656 µs (26.3%)
appsec_no_iast	1.727 ms [1.703 ms, 1.751 ms]	361.998 µs (26.5%)
code_origins	1.679 ms [1.647 ms, 1.711 ms]	313.975 µs (23.0%)
iast	1.511 ms [1.487 ms, 1.536 ms]	146.468 µs (10.7%)
profiling	1.556 ms [1.53 ms, 1.582 ms]	191.164 µs (14.0%)
tracing	1.476 ms [1.452 ms, 1.499 ms]	110.588 µs (8.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.36 ms [1.34 ms, 1.38 ms]	-
appsec	1.738 ms [1.715 ms, 1.761 ms]	377.521 µs (27.8%)
appsec_no_iast	1.73 ms [1.705 ms, 1.755 ms]	369.46 µs (27.2%)
code_origins	1.704 ms [1.671 ms, 1.736 ms]	343.391 µs (25.2%)
iast	1.514 ms [1.491 ms, 1.538 ms]	154.14 µs (11.3%)
profiling	1.564 ms [1.54 ms, 1.588 ms]	203.748 µs (15.0%)
tracing	1.521 ms [1.494 ms, 1.547 ms]	160.253 µs (11.8%)

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (381.252 µs) : 361, 401
. : milestone, 381,
iast (512.711 µs) : 491, 535
. : milestone, 513,
iast_FULL (730.087 µs) : 708, 752
. : milestone, 730,
iast_GLOBAL (560.033 µs) : 538, 582
. : milestone, 560,
iast_HARDCODED_SECRET_DISABLED (507.758 µs) : 486, 530
. : milestone, 508,
iast_INACTIVE (460.525 µs) : 439, 482
. : milestone, 461,
iast_TELEMETRY_OFF (509.402 µs) : 487, 532
. : milestone, 509,
tracing (455.255 µs) : 434, 476
. : milestone, 455,
section candidate
no_agent (380.74 µs) : 360, 401
. : milestone, 381,
iast (510.073 µs) : 488, 532
. : milestone, 510,
iast_FULL (728.689 µs) : 707, 751
. : milestone, 729,
iast_GLOBAL (566.248 µs) : 544, 589
. : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (512.357 µs) : 490, 535
. : milestone, 512,
iast_INACTIVE (463.777 µs) : 442, 486
. : milestone, 464,
iast_TELEMETRY_OFF (503.931 µs) : 481, 527
. : milestone, 504,
tracing (457.634 µs) : 437, 479
. : milestone, 458,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.252 µs [361.415 µs, 401.09 µs]	-
iast	512.711 µs [490.816 µs, 534.605 µs]	131.459 µs (34.5%)
iast_FULL	730.087 µs [708.02 µs, 752.153 µs]	348.835 µs (91.5%)
iast_GLOBAL	560.033 µs [538.302 µs, 581.765 µs]	178.781 µs (46.9%)
iast_HARDCODED_SECRET_DISABLED	507.758 µs [485.869 µs, 529.647 µs]	126.505 µs (33.2%)
iast_INACTIVE	460.525 µs [438.908 µs, 482.141 µs]	79.273 µs (20.8%)
iast_TELEMETRY_OFF	509.402 µs [487.125 µs, 531.679 µs]	128.15 µs (33.6%)
tracing	455.255 µs [434.484 µs, 476.026 µs]	74.003 µs (19.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	380.74 µs [360.193 µs, 401.287 µs]	-
iast	510.073 µs [488.262 µs, 531.885 µs]	129.333 µs (34.0%)
iast_FULL	728.689 µs [706.562 µs, 750.816 µs]	347.948 µs (91.4%)
iast_GLOBAL	566.248 µs [543.515 µs, 588.982 µs]	185.508 µs (48.7%)
iast_HARDCODED_SECRET_DISABLED	512.357 µs [489.829 µs, 534.885 µs]	131.617 µs (34.6%)
iast_INACTIVE	463.777 µs [441.785 µs, 485.77 µs]	83.037 µs (21.8%)
iast_TELEMETRY_OFF	503.931 µs [481.213 µs, 526.649 µs]	123.191 µs (32.4%)
tracing	457.634 µs [436.582 µs, 478.685 µs]	76.893 µs (20.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-env-08e0fa3c435603d3b69ac98b0120f4507a356d0a
git_commit_date	1741082792	1740744591
git_commit_sha	`8f945b0`	`08e0fa3`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~08e0fa3c43

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1741091151	1741091151
ci_job_id	831517658	831517658
ci_pipeline_id	57626459	57626459
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-xk9phren-project-304-concurrent-1-hilw58g5 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-xk9phren-project-304-concurrent-1-hilw58g5 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (14.907 s) : 14907000, 14907000
. : milestone, 14907000,
appsec (14.731 s) : 14731000, 14731000
. : milestone, 14731000,
iast (18.97 s) : 18970000, 18970000
. : milestone, 18970000,
iast_GLOBAL (17.961 s) : 17961000, 17961000
. : milestone, 17961000,
profiling (15.14 s) : 15140000, 15140000
. : milestone, 15140000,
tracing (14.951 s) : 14951000, 14951000
. : milestone, 14951000,
section candidate
no_agent (15.259 s) : 15259000, 15259000
. : milestone, 15259000,
appsec (15.159 s) : 15159000, 15159000
. : milestone, 15159000,
iast (18.989 s) : 18989000, 18989000
. : milestone, 18989000,
iast_GLOBAL (18.049 s) : 18049000, 18049000
. : milestone, 18049000,
profiling (15.795 s) : 15795000, 15795000
. : milestone, 15795000,
tracing (14.901 s) : 14901000, 14901000
. : milestone, 14901000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.907 s [14.907 s, 14.907 s]	-
appsec	14.731 s [14.731 s, 14.731 s]	-176.0 ms (-1.2%)
iast	18.97 s [18.97 s, 18.97 s]	4.063 s (27.3%)
iast_GLOBAL	17.961 s [17.961 s, 17.961 s]	3.054 s (20.5%)
profiling	15.14 s [15.14 s, 15.14 s]	233.0 ms (1.6%)
tracing	14.951 s [14.951 s, 14.951 s]	44.0 ms (0.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.259 s [15.259 s, 15.259 s]	-
appsec	15.159 s [15.159 s, 15.159 s]	-100.0 ms (-0.7%)
iast	18.989 s [18.989 s, 18.989 s]	3.73 s (24.4%)
iast_GLOBAL	18.049 s [18.049 s, 18.049 s]	2.79 s (18.3%)
profiling	15.795 s [15.795 s, 15.795 s]	536.0 ms (3.5%)
tracing	14.901 s [14.901 s, 14.901 s]	-358.0 ms (-2.3%)

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~08e0fa3c43, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
. : milestone, 1476,
appsec (2.355 ms) : 2312, 2399
. : milestone, 2355,
iast (2.121 ms) : 2065, 2176
. : milestone, 2121,
iast_GLOBAL (2.17 ms) : 2114, 2226
. : milestone, 2170,
profiling (1.976 ms) : 1932, 2020
. : milestone, 1976,
tracing (1.955 ms) : 1913, 1997
. : milestone, 1955,
section candidate
no_agent (1.483 ms) : 1471, 1495
. : milestone, 1483,
appsec (2.359 ms) : 2316, 2403
. : milestone, 2359,
iast (2.12 ms) : 2065, 2176
. : milestone, 2120,
iast_GLOBAL (2.165 ms) : 2109, 2220
. : milestone, 2165,
profiling (2.005 ms) : 1961, 2050
. : milestone, 2005,
tracing (1.948 ms) : 1906, 1990
. : milestone, 1948,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.355 ms [2.312 ms, 2.399 ms]	879.441 µs (59.6%)
iast	2.121 ms [2.065 ms, 2.176 ms]	644.626 µs (43.7%)
iast_GLOBAL	2.17 ms [2.114 ms, 2.226 ms]	694.112 µs (47.0%)
profiling	1.976 ms [1.932 ms, 2.02 ms]	500.278 µs (33.9%)
tracing	1.955 ms [1.913 ms, 1.997 ms]	479.15 µs (32.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.483 ms [1.471 ms, 1.495 ms]	-
appsec	2.359 ms [2.316 ms, 2.403 ms]	876.476 µs (59.1%)
iast	2.12 ms [2.065 ms, 2.176 ms]	637.45 µs (43.0%)
iast_GLOBAL	2.165 ms [2.109 ms, 2.22 ms]	681.591 µs (46.0%)
profiling	2.005 ms [1.961 ms, 2.05 ms]	522.443 µs (35.2%)
tracing	1.948 ms [1.906 ms, 1.99 ms]	465.286 µs (31.4%)

nikita-tkachenko-datadogand others added 2 commits February 28, 2025 11:44

Implement tests reordering for TestNG (#8467)
5328e2b

Don't skip muzzle and test_published_artifacts for the nightly (#8449)
08e0fa3

nikita-tkachenko-datadog added the tag: do not merge Do not merge changes label Mar 4, 2025

nikita-tkachenko-datadog closed this Mar 4, 2025

nikita-tkachenko-datadog deleted the nikita-tkachenko/test-env-08e0fa3c435603d3b69ac98b0120f4507a356d0a branch March 4, 2025 14:10

Navigation Menu

Uh oh!

Nikita tkachenko/test env 08e0fa3c435603d3b69ac98b0120f4507a356d0a - #8498

Nikita tkachenko/test env 08e0fa3c435603d3b69ac98b0120f4507a356d0a#8498
nikita-tkachenko-datadog wants to merge 2 commits into
nikita-tkachenko/test-env-72fbf84f74d6af5824ed6f6dc5539d82068a78edfrom
nikita-tkachenko/test-env-08e0fa3c435603d3b69ac98b0120f4507a356d0a

nikita-tkachenko-datadog commented Mar 4, 2025

Uh oh!

pr-commenterBot commented Mar 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

nikita-tkachenko-datadog commented Mar 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Mar 4, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants