Nikita tkachenko/test env 19d8832a9d074b4e378be52f31eb1627522341a6 by nikita-tkachenko-datadog · Pull Request #8495 · DataDog/dd-trace-java

nikita-tkachenko-datadog · 2025-03-04T11:57:18Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

* Add checks, spotless, and test_published_artifact * small fixes * WIP * syntax errors * add gradle daemon logs everywhere * fix c/p error * fix system tests dir * add some ls * always save artifacts * fix workspace path for system tests * move more jobs to the tests stage * split muzzle into 6 jobs * remove system-tests debug output * fix path and index for muzzle * add signing args * 8 parallism * no needs for spotless * java 7 for test published artifacts * use the image for the test jvm * upload ci_app results * when always -> on_success * Cleanup TraceAgentTests to always run * try agent integration test * add the actual target * clean up branch * set keys and use explicit agent value in test * remove DD_* variables from environment before tests * save/restore must be toplevel * some debug logic for keys * yaml * set api keys in after_script * fix DDApiIntegrationTest * remove agent integration tests from circleci * cleanup merge issues * Revert "remove agent integration tests from circleci" This reverts commit 90602a2. * assign CI_AGENT_HOST to localhost * don't skip tests when populating cache * codenarc

…ns (#8451)

pr-commenter · 2025-03-04T12:36:27Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6
git_commit_date	1741082792	1740758111
git_commit_sha	`8f945b0`	`19d8832`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~19d8832a9d

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741091740	1741091740
ci_job_id	831518151	831518151
ci_pipeline_id	57626491	57626491
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner--t2f38de-project-304-concurrent-7-5dhvbaz4 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner--t2f38de-project-304-concurrent-7-5dhvbaz4 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 21 performance improvements and 5 performance regressions! Performance is the same for 27 metrics, 10 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent	better [-691.701ms; -493.437ms] or [-7.006%; -4.998%]	9.280s	9.873s
scenario:startup:insecure-bank:iast:Agent.start	better [-92.711ms; -80.126ms] or [-7.368%; -6.368%]	1.172s	1.258s
scenario:startup:insecure-bank:iast:Remote Config	worse [+90.882µs; +112.410µs] or [+17.735%; +21.936%]	614.082µs	512.436µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent	better [-796.981ms; -583.821ms] or [-8.066%; -5.909%]	9.190s	9.881s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start	better [-110.577ms; -69.180ms] or [-8.719%; -5.455%]	1.178s	1.268s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent	better [-711.059ms; -619.398ms] or [-7.200%; -6.272%]	9.211s	9.876s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start	better [-91.899ms; -60.858ms] or [-7.352%; -4.869%]	1.174s	1.250s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config	worse [+91.409µs; +119.490µs] or [+18.078%; +23.631%]	611.096µs	505.647µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry	better [-3.436ms; -3.230ms] or [-28.772%; -27.045%]	8.610ms	11.943ms
scenario:startup:insecure-bank:tracing:Agent	better [-681.956ms; -580.134ms] or [-7.349%; -6.252%]	8.649s	9.280s
scenario:startup:insecure-bank:tracing:Agent.start	better [-111.782ms; -88.975ms] or [-9.821%; -7.817%]	1.038s	1.138s
scenario:startup:petclinic:appsec:Agent	better [-902.823ms; -791.896ms] or [-7.801%; -6.843%]	10.726s	11.573s
scenario:startup:petclinic:appsec:Agent.start	better [-104.176ms; -83.728ms] or [-8.180%; -6.574%]	1.180s	1.274s
scenario:startup:petclinic:appsec:Remote Config	worse [+17.140µs; +68.178µs] or [+2.717%; +10.809%]	673.400µs	630.741µs
scenario:startup:petclinic:appsec:Telemetry	better [-4.230ms; -3.840ms] or [-34.304%; -31.136%]	8.297ms	12.332ms
scenario:startup:petclinic:iast:Agent	better [-1091.235ms; -673.057ms] or [-9.120%; -5.625%]	11.083s	11.965s
scenario:startup:petclinic:iast:Agent.start	better [-99.770ms; -82.097ms] or [-7.899%; -6.500%]	1.172s	1.263s
scenario:startup:petclinic:iast:Remote Config	worse [+67.226µs; +116.903µs] or [+12.930%; +22.485%]	611.987µs	519.922µs
scenario:startup:petclinic:profiling:Agent	better [-987.017ms; -832.638ms] or [-8.401%; -7.087%]	10.839s	11.749s
scenario:startup:petclinic:profiling:Agent.start	better [-138.802ms; -91.312ms] or [-10.004%; -6.581%]	1.272s	1.387s
scenario:startup:petclinic:profiling:ProfilingAgent	better [-29.519ms; -25.140ms] or [-23.821%; -20.288%]	96.589ms	123.919ms
scenario:startup:petclinic:profiling:Telemetry	better [-4.028ms; -3.465ms] or [-31.418%; -27.024%]	9.075ms	12.822ms
scenario:startup:petclinic:profiling:Profiling	better [-29.516ms; -25.137ms] or [-23.815%; -20.281%]	96.613ms	123.940ms
scenario:startup:petclinic:tracing:Agent	better [-1.202s; -1.023s] or [-10.486%; -8.928%]	10.347s	11.459s
scenario:startup:petclinic:tracing:Agent.start	better [-99.406ms; -89.659ms] or [-8.767%; -7.908%]	1.039s	1.134s
scenario:startup:petclinic:tracing:Remote Config	worse [+91.328µs; +139.681µs] or [+15.651%; +23.938%]	699.018µs	583.513µs

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.138 s) : 0, 1138216
Total [baseline] (9.28 s) : 0, 9279883
Agent [candidate] (1.038 s) : 0, 1037838
Total [candidate] (8.649 s) : 0, 8648838
section iast
Agent [baseline] (1.258 s) : 0, 1258352
Total [baseline] (9.873 s) : 0, 9872689
Agent [candidate] (1.172 s) : 0, 1171934
Total [candidate] (9.28 s) : 0, 9280120
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.268 s) : 0, 1268303
Total [baseline] (9.881 s) : 0, 9880769
Agent [candidate] (1.178 s) : 0, 1178424
Total [candidate] (9.19 s) : 0, 9190368
section iast_TELEMETRY_OFF
Agent [baseline] (1.25 s) : 0, 1249976
Total [baseline] (9.876 s) : 0, 9875844
Agent [candidate] (1.174 s) : 0, 1173598
Total [candidate] (9.211 s) : 0, 9210616

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.138 s	-
Agent	iast	1.258 s	120.136 ms (10.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.268 s	130.086 ms (11.4%)
Agent	iast_TELEMETRY_OFF	1.25 s	111.76 ms (9.8%)
Total	tracing	9.28 s	-
Total	iast	9.873 s	592.806 ms (6.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.881 s	600.886 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.876 s	595.961 ms (6.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.038 s	-
Agent	iast	1.172 s	134.096 ms (12.9%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.178 s	140.586 ms (13.5%)
Agent	iast_TELEMETRY_OFF	1.174 s	135.759 ms (13.1%)
Total	tracing	8.649 s	-
Total	iast	9.28 s	631.282 ms (7.3%)
Total	iast_HARDCODED_SECRET_DISABLED	9.19 s	541.53 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.211 s	561.778 ms (6.5%)

gantt
title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.665 ms) : 0, 720665
BytebuddyAgent [candidate] (716.354 ms) : 0, 716354
GlobalTracer [baseline] (239.521 ms) : 0, 239521
GlobalTracer [candidate] (239.005 ms) : 0, 239005
AppSec [baseline] (54.892 ms) : 0, 54892
AppSec [candidate] (55.538 ms) : 0, 55538
Remote Config [baseline] (593.546 µs) : 0, 594
Remote Config [candidate] (708.667 µs) : 0, 709
Telemetry [baseline] (12.327 ms) : 0, 12327
Telemetry [candidate] (11.418 ms) : 0, 11418
Debugger [baseline] (95.105 ms) : 0, 95105
section iast
BytebuddyAgent [baseline] (835.61 ms) : 0, 835610
BytebuddyAgent [candidate] (836.417 ms) : 0, 836417
GlobalTracer [baseline] (230.396 ms) : 0, 230396
GlobalTracer [candidate] (230.601 ms) : 0, 230601
IAST [baseline] (22.859 ms) : 0, 22859
IAST [candidate] (23.185 ms) : 0, 23185
AppSec [baseline] (56.954 ms) : 0, 56954
AppSec [candidate] (57.5 ms) : 0, 57500
Remote Config [baseline] (512.436 µs) : 0, 512
Remote Config [candidate] (614.082 µs) : 0, 614
Telemetry [baseline] (13.589 ms) : 0, 13589
Telemetry [candidate] (8.698 ms) : 0, 8698
Debugger [baseline] (83.499 ms) : 0, 83499
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (842.129 ms) : 0, 842129
BytebuddyAgent [candidate] (843.142 ms) : 0, 843142
GlobalTracer [baseline] (231.979 ms) : 0, 231979
GlobalTracer [candidate] (230.234 ms) : 0, 230234
IAST [baseline] (23.162 ms) : 0, 23162
IAST [candidate] (22.885 ms) : 0, 22885
AppSec [baseline] (57.547 ms) : 0, 57547
AppSec [candidate] (57.647 ms) : 0, 57647
Remote Config [baseline] (521.511 µs) : 0, 522
Remote Config [candidate] (615.098 µs) : 0, 615
Telemetry [baseline] (13.732 ms) : 0, 13732
Telemetry [candidate] (8.797 ms) : 0, 8797
Debugger [baseline] (84.173 ms) : 0, 84173
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.993 ms) : 0, 830993
BytebuddyAgent [candidate] (839.398 ms) : 0, 839398
GlobalTracer [baseline] (229.242 ms) : 0, 229242
GlobalTracer [candidate] (230.751 ms) : 0, 230751
IAST [baseline] (22.267 ms) : 0, 22267
IAST [candidate] (25.504 ms) : 0, 25504
AppSec [baseline] (56.733 ms) : 0, 56733
AppSec [candidate] (53.722 ms) : 0, 53722
Remote Config [baseline] (505.647 µs) : 0, 506
Remote Config [candidate] (611.096 µs) : 0, 611
Telemetry [baseline] (11.943 ms) : 0, 11943
Telemetry [candidate] (8.61 ms) : 0, 8610
Debugger [baseline] (83.338 ms) : 0, 83338

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.134 s) : 0, 1133850
Total [baseline] (11.459 s) : 0, 11459279
Agent [candidate] (1.039 s) : 0, 1039318
Total [candidate] (10.347 s) : 0, 10346927
section appsec
Agent [baseline] (1.274 s) : 0, 1273597
Total [baseline] (11.573 s) : 0, 11573066
Agent [candidate] (1.18 s) : 0, 1179645
Total [candidate] (10.726 s) : 0, 10725706
section iast
Agent [baseline] (1.263 s) : 0, 1263064
Total [baseline] (11.965 s) : 0, 11964960
Agent [candidate] (1.172 s) : 0, 1172130
Total [candidate] (11.083 s) : 0, 11082814
section profiling
Agent [baseline] (1.387 s) : 0, 1387484
Total [baseline] (11.749 s) : 0, 11748788
Agent [candidate] (1.272 s) : 0, 1272427
Total [candidate] (10.839 s) : 0, 10838961

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.134 s	-
Agent	appsec	1.274 s	139.747 ms (12.3%)
Agent	iast	1.263 s	129.214 ms (11.4%)
Agent	profiling	1.387 s	253.634 ms (22.4%)
Total	tracing	11.459 s	-
Total	appsec	11.573 s	113.787 ms (1.0%)
Total	iast	11.965 s	505.681 ms (4.4%)
Total	profiling	11.749 s	289.509 ms (2.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	appsec	1.18 s	140.328 ms (13.5%)
Agent	iast	1.172 s	132.812 ms (12.8%)
Agent	profiling	1.272 s	233.11 ms (22.4%)
Total	tracing	10.347 s	-
Total	appsec	10.726 s	378.779 ms (3.7%)
Total	iast	11.083 s	735.887 ms (7.1%)
Total	profiling	10.839 s	492.034 ms (4.8%)

gantt
title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.69 ms) : 0, 715690
BytebuddyAgent [candidate] (717.27 ms) : 0, 717270
GlobalTracer [baseline] (239.429 ms) : 0, 239429
GlobalTracer [candidate] (239.223 ms) : 0, 239223
AppSec [baseline] (55.518 ms) : 0, 55518
AppSec [candidate] (55.721 ms) : 0, 55721
Remote Config [baseline] (583.513 µs) : 0, 584
Remote Config [candidate] (699.018 µs) : 0, 699
Telemetry [baseline] (12.34 ms) : 0, 12340
Telemetry [candidate] (11.489 ms) : 0, 11489
Debugger [baseline] (95.284 ms) : 0, 95284
section appsec
BytebuddyAgent [baseline] (736.116 ms) : 0, 736116
BytebuddyAgent [candidate] (732.736 ms) : 0, 732736
GlobalTracer [baseline] (236.921 ms) : 0, 236921
GlobalTracer [candidate] (235.291 ms) : 0, 235291
AppSec [baseline] (177.493 ms) : 0, 177493
AppSec [candidate] (176.972 ms) : 0, 176972
Remote Config [baseline] (630.741 µs) : 0, 631
Remote Config [candidate] (673.4 µs) : 0, 673
Telemetry [baseline] (12.332 ms) : 0, 12332
Telemetry [candidate] (8.297 ms) : 0, 8297
Debugger [baseline] (84.469 ms) : 0, 84469
IAST [baseline] (21.275 ms) : 0, 21275
IAST [candidate] (21.561 ms) : 0, 21561
section iast
BytebuddyAgent [baseline] (836.073 ms) : 0, 836073
BytebuddyAgent [candidate] (836.354 ms) : 0, 836354
GlobalTracer [baseline] (230.524 ms) : 0, 230524
GlobalTracer [candidate] (230.33 ms) : 0, 230330
AppSec [baseline] (56.699 ms) : 0, 56699
AppSec [candidate] (57.746 ms) : 0, 57746
Remote Config [baseline] (519.922 µs) : 0, 520
Remote Config [candidate] (611.987 µs) : 0, 612
Telemetry [baseline] (16.707 ms) : 0, 16707
Telemetry [candidate] (8.913 ms) : 0, 8913
Debugger [baseline] (83.784 ms) : 0, 83784
IAST [baseline] (23.729 ms) : 0, 23729
IAST [candidate] (23.171 ms) : 0, 23171
section profiling
ProfilingAgent [baseline] (123.919 ms) : 0, 123919
ProfilingAgent [candidate] (96.589 ms) : 0, 96589
BytebuddyAgent [baseline] (712.933 ms) : 0, 712933
BytebuddyAgent [candidate] (719.052 ms) : 0, 719052
GlobalTracer [baseline] (352.333 ms) : 0, 352333
GlobalTracer [candidate] (351.428 ms) : 0, 351428
AppSec [baseline] (55.495 ms) : 0, 55495
AppSec [candidate] (54.644 ms) : 0, 54644
Remote Config [baseline] (624.06 µs) : 0, 624
Remote Config [candidate] (671.738 µs) : 0, 672
Telemetry [baseline] (12.822 ms) : 0, 12822
Telemetry [candidate] (9.075 ms) : 0, 9075
Profiling [baseline] (123.94 ms) : 0, 123940
Profiling [candidate] (96.613 ms) : 0, 96613
Debugger [baseline] (88.737 ms) : 0, 88737

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-04T12:04:57	2025-03-04T12:12:48
git_branch	master	nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6
git_commit_date	1741082792	1740758111
git_commit_sha	`8f945b0`	`19d8832`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~19d8832a9d
start_time	2025-03-04T12:04:43	2025-03-04T12:12:34

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741090765	1741090765
ci_job_id	831518152	831518152
ci_pipeline_id	57626491	57626491
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8ksbllmc-project-304-concurrent-5-eejg7wux 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8ksbllmc-project-304-concurrent-5-eejg7wux 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (1.338 ms) : 1318, 1358
. : milestone, 1338,
appsec (1.729 ms) : 1705, 1752
. : milestone, 1729,
appsec_no_iast (1.725 ms) : 1701, 1749
. : milestone, 1725,
code_origins (1.686 ms) : 1653, 1718
. : milestone, 1686,
iast (1.504 ms) : 1479, 1529
. : milestone, 1504,
profiling (1.507 ms) : 1482, 1531
. : milestone, 1507,
tracing (1.482 ms) : 1456, 1509
. : milestone, 1482,
section candidate
no_agent (1.343 ms) : 1323, 1363
. : milestone, 1343,
appsec (1.718 ms) : 1693, 1742
. : milestone, 1718,
appsec_no_iast (1.727 ms) : 1702, 1752
. : milestone, 1727,
code_origins (1.684 ms) : 1650, 1718
. : milestone, 1684,
iast (1.507 ms) : 1483, 1531
. : milestone, 1507,
profiling (1.56 ms) : 1534, 1586
. : milestone, 1560,
tracing (1.472 ms) : 1447, 1496
. : milestone, 1472,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.338 ms [1.318 ms, 1.358 ms]	-
appsec	1.729 ms [1.705 ms, 1.752 ms]	391.063 µs (29.2%)
appsec_no_iast	1.725 ms [1.701 ms, 1.749 ms]	387.643 µs (29.0%)
code_origins	1.686 ms [1.653 ms, 1.718 ms]	348.096 µs (26.0%)
iast	1.504 ms [1.479 ms, 1.529 ms]	166.344 µs (12.4%)
profiling	1.507 ms [1.482 ms, 1.531 ms]	169.04 µs (12.6%)
tracing	1.482 ms [1.456 ms, 1.509 ms]	144.649 µs (10.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.343 ms [1.323 ms, 1.363 ms]	-
appsec	1.718 ms [1.693 ms, 1.742 ms]	374.591 µs (27.9%)
appsec_no_iast	1.727 ms [1.702 ms, 1.752 ms]	384.146 µs (28.6%)
code_origins	1.684 ms [1.65 ms, 1.718 ms]	340.884 µs (25.4%)
iast	1.507 ms [1.483 ms, 1.531 ms]	163.623 µs (12.2%)
profiling	1.56 ms [1.534 ms, 1.586 ms]	217.264 µs (16.2%)
tracing	1.472 ms [1.447 ms, 1.496 ms]	128.581 µs (9.6%)

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (375.865 µs) : 356, 396
. : milestone, 376,
iast (507.78 µs) : 486, 530
. : milestone, 508,
iast_FULL (719.903 µs) : 698, 742
. : milestone, 720,
iast_GLOBAL (555.673 µs) : 533, 578
. : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (507.917 µs) : 486, 530
. : milestone, 508,
iast_INACTIVE (458.795 µs) : 437, 480
. : milestone, 459,
iast_TELEMETRY_OFF (501.207 µs) : 479, 524
. : milestone, 501,
tracing (459.403 µs) : 437, 481
. : milestone, 459,
section candidate
no_agent (371.728 µs) : 352, 391
. : milestone, 372,
iast (510.085 µs) : 488, 532
. : milestone, 510,
iast_FULL (718.68 µs) : 697, 741
. : milestone, 719,
iast_GLOBAL (558.483 µs) : 536, 581
. : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (511.51 µs) : 489, 534
. : milestone, 512,
iast_INACTIVE (455.336 µs) : 434, 476
. : milestone, 455,
iast_TELEMETRY_OFF (494.289 µs) : 472, 517
. : milestone, 494,
tracing (451.395 µs) : 430, 473
. : milestone, 451,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	375.865 µs [355.712 µs, 396.019 µs]	-
iast	507.78 µs [485.842 µs, 529.717 µs]	131.914 µs (35.1%)
iast_FULL	719.903 µs [697.754 µs, 742.053 µs]	344.038 µs (91.5%)
iast_GLOBAL	555.673 µs [533.47 µs, 577.876 µs]	179.808 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED	507.917 µs [486.172 µs, 529.662 µs]	132.052 µs (35.1%)
iast_INACTIVE	458.795 µs [437.24 µs, 480.35 µs]	82.93 µs (22.1%)
iast_TELEMETRY_OFF	501.207 µs [478.634 µs, 523.781 µs]	125.342 µs (33.3%)
tracing	459.403 µs [437.361 µs, 481.444 µs]	83.537 µs (22.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	371.728 µs [352.154 µs, 391.301 µs]	-
iast	510.085 µs [487.951 µs, 532.219 µs]	138.357 µs (37.2%)
iast_FULL	718.68 µs [696.705 µs, 740.656 µs]	346.953 µs (93.3%)
iast_GLOBAL	558.483 µs [536.024 µs, 580.942 µs]	186.755 µs (50.2%)
iast_HARDCODED_SECRET_DISABLED	511.51 µs [488.884 µs, 534.136 µs]	139.782 µs (37.6%)
iast_INACTIVE	455.336 µs [434.493 µs, 476.18 µs]	83.609 µs (22.5%)
iast_TELEMETRY_OFF	494.289 µs [471.669 µs, 516.91 µs]	122.561 µs (33.0%)
tracing	451.395 µs [430.212 µs, 472.579 µs]	79.667 µs (21.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6
git_commit_date	1741082792	1740758111
git_commit_sha	`8f945b0`	`19d8832`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~19d8832a9d

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1741091279	1741091279
ci_job_id	831518153	831518153
ci_pipeline_id	57626491	57626491
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner--t2f38de-project-304-concurrent-8-kbmn207h 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner--t2f38de-project-304-concurrent-8-kbmn207h 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (15.008 s) : 15008000, 15008000
. : milestone, 15008000,
appsec (15.083 s) : 15083000, 15083000
. : milestone, 15083000,
iast (19.208 s) : 19208000, 19208000
. : milestone, 19208000,
iast_GLOBAL (18.172 s) : 18172000, 18172000
. : milestone, 18172000,
profiling (15.041 s) : 15041000, 15041000
. : milestone, 15041000,
tracing (15.015 s) : 15015000, 15015000
. : milestone, 15015000,
section candidate
no_agent (15.426 s) : 15426000, 15426000
. : milestone, 15426000,
appsec (15.002 s) : 15002000, 15002000
. : milestone, 15002000,
iast (18.847 s) : 18847000, 18847000
. : milestone, 18847000,
iast_GLOBAL (18.113 s) : 18113000, 18113000
. : milestone, 18113000,
profiling (15.105 s) : 15105000, 15105000
. : milestone, 15105000,
tracing (14.663 s) : 14663000, 14663000
. : milestone, 14663000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.008 s [15.008 s, 15.008 s]	-
appsec	15.083 s [15.083 s, 15.083 s]	75.0 ms (0.5%)
iast	19.208 s [19.208 s, 19.208 s]	4.2 s (28.0%)
iast_GLOBAL	18.172 s [18.172 s, 18.172 s]	3.164 s (21.1%)
profiling	15.041 s [15.041 s, 15.041 s]	33.0 ms (0.2%)
tracing	15.015 s [15.015 s, 15.015 s]	7.0 ms (0.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.426 s [15.426 s, 15.426 s]	-
appsec	15.002 s [15.002 s, 15.002 s]	-424.0 ms (-2.7%)
iast	18.847 s [18.847 s, 18.847 s]	3.421 s (22.2%)
iast_GLOBAL	18.113 s [18.113 s, 18.113 s]	2.687 s (17.4%)
profiling	15.105 s [15.105 s, 15.105 s]	-321.0 ms (-2.1%)
tracing	14.663 s [14.663 s, 14.663 s]	-763.0 ms (-4.9%)

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
dateFormat X
axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
. : milestone, 1469,
appsec (2.319 ms) : 2275, 2362
. : milestone, 2319,
iast (2.112 ms) : 2056, 2167
. : milestone, 2112,
iast_GLOBAL (2.148 ms) : 2092, 2204
. : milestone, 2148,
profiling (1.961 ms) : 1917, 2004
. : milestone, 1961,
tracing (1.946 ms) : 1904, 1989
. : milestone, 1946,
section candidate
no_agent (1.469 ms) : 1458, 1481
. : milestone, 1469,
appsec (2.325 ms) : 2282, 2369
. : milestone, 2325,
iast (2.113 ms) : 2058, 2169
. : milestone, 2113,
iast_GLOBAL (2.148 ms) : 2093, 2204
. : milestone, 2148,
profiling (1.981 ms) : 1936, 2026
. : milestone, 1981,
tracing (1.947 ms) : 1904, 1989
. : milestone, 1947,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.458 ms, 1.481 ms]	-
appsec	2.319 ms [2.275 ms, 2.362 ms]	849.267 µs (57.8%)
iast	2.112 ms [2.056 ms, 2.167 ms]	642.355 µs (43.7%)
iast_GLOBAL	2.148 ms [2.092 ms, 2.204 ms]	678.579 µs (46.2%)
profiling	1.961 ms [1.917 ms, 2.004 ms]	491.333 µs (33.4%)
tracing	1.946 ms [1.904 ms, 1.989 ms]	476.915 µs (32.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.458 ms, 1.481 ms]	-
appsec	2.325 ms [2.282 ms, 2.369 ms]	856.145 µs (58.3%)
iast	2.113 ms [2.058 ms, 2.169 ms]	644.125 µs (43.8%)
iast_GLOBAL	2.148 ms [2.093 ms, 2.204 ms]	679.186 µs (46.2%)
profiling	1.981 ms [1.936 ms, 2.026 ms]	511.594 µs (34.8%)
tracing	1.947 ms [1.904 ms, 1.989 ms]	477.256 µs (32.5%)

randomandersonand others added 2 commits February 28, 2025 09:44

Use separate TestEventHandlers per framework in CI Vis instrumentatio…
19d8832
…ns (#8451)

nikita-tkachenko-datadog added the tag: do not merge Do not merge changes label Mar 4, 2025

nikita-tkachenko-datadog closed this Mar 4, 2025

nikita-tkachenko-datadog deleted the nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6 branch March 4, 2025 14:10

Navigation Menu

Uh oh!

Nikita tkachenko/test env 19d8832a9d074b4e378be52f31eb1627522341a6 - #8495

Nikita tkachenko/test env 19d8832a9d074b4e378be52f31eb1627522341a6#8495
nikita-tkachenko-datadog wants to merge 2 commits into
nikita-tkachenko/test-env-87b536644c4927894b6b24d99c5088a787f44deafrom
nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6

nikita-tkachenko-datadog commented Mar 4, 2025

Uh oh!

pr-commenterBot commented Mar 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

nikita-tkachenko-datadog commented Mar 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Mar 4, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants