Add In-Product Enablement by jpbempel · Pull Request #8461 · DataDog/dd-trace-java

jpbempel · 2025-02-27T13:22:09Z

What Does This Do

Introduce the ability to start and stop Debugger features:

Dynamic Instrumentation
Exception Replay
Code Origin
Distributed Debugger dynamically based on RemoteConfig record: APM_TRACING DebuggerAgent is now run every time at startup to have the base of some feature ready and be able to start the minimum required foe each feature.
Ability to stop also the feature at any time to uninstall probes. Add smoke tests

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: DEBUG-3455

pr-commenter · 2025-02-27T13:43:20Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1740759517	1740759894
end_time	2025-02-28T16:19:54	2025-02-28T16:26:11
git_branch	master	jpbempel/in-product-enablement
git_commit_sha	`19d8832`	`676ddbb`
start_time	2025-02-28T16:18:38	2025-02-28T16:24:55

See matching parameters

	Baseline	Candidate
ci_job_id	828044660	828044660
ci_pipeline_id	57360274	57360274
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1740758874	1740758874

Summary

Found 0 performance improvements and 4 performance regressions! Performance is the same for 5 metrics, 6 unstable metrics.

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:loop	worse [+277.814µs; +372.102µs] or [+2.697%; +3.612%]	worse [+257.613µs; +373.795µs] or [+2.456%; +3.563%]	worse [+232.602µs; +370.545µs] or [+2.198%; +3.501%]	unsure [+42.856µs; +657.330µs] or [+0.391%; +5.993%]	worse [-3.439op/s; -1.754op/s] or [-3.646%; -1.859%]

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-15.307µs; +23.218µs] or [-5.739%; +8.705%]	unstable [-25.100µs; +35.501µs] or [-8.232%; +11.644%]	unstable [-35.505µs; +46.406µs] or [-11.144%; +14.565%]	unstable [-91.109µs; +86.280µs] or [-15.830%; +14.990%]	same
scenario:basic	same	same	same	unstable [-37.764µs; +20.984µs] or [-7.381%; +4.101%]	unstable [-199.866op/s; +64.913op/s] or [-7.595%; +2.467%]

Request duration reports for reports

gantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (304.887 µs) : 277, 333
. : milestone, 305,
basic (296.286 µs) : 288, 304
. : milestone, 296,
loop (10.49 ms) : 10419, 10561
. : milestone, 10490,
section candidate
noprobe (310.087 µs) : 280, 340
. : milestone, 310,
basic (301.477 µs) : 293, 310
. : milestone, 301,
loop (10.805 ms) : 10778, 10833
. : milestone, 10805,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	304.887 µs [277.089 µs, 332.684 µs]
basic	296.286 µs [288.452 µs, 304.12 µs]
loop	10.49 ms [10.419 ms, 10.561 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	310.087 µs [280.16 µs, 340.015 µs]
basic	301.477 µs [292.605 µs, 310.35 µs]
loop	10.805 ms [10.778 ms, 10.833 ms]

pr-commenter · 2025-02-27T14:04:55Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/in-product-enablement
git_commit_date	1740758111	1740758874
git_commit_sha	`19d8832`	`676ddbb`
release_version	1.47.0-SNAPSHOT~19d8832a9d	1.47.0-SNAPSHOT~676ddbb84a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1740761325	1740761325
ci_job_id	828044652	828044652
ci_pipeline_id	57360274	57360274
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8g9zkp7x-project-304-concurrent-0-yj40hyj8 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8g9zkp7x-project-304-concurrent-0-yj40hyj8 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 6 performance improvements and 24 performance regressions! Performance is the same for 27 metrics, 6 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent	worse [+501.772ms; +666.016ms] or [+5.402%; +7.170%]	9.872s	9.288s
scenario:startup:insecure-bank:iast:Agent.start	worse [+85.352ms; +94.648ms] or [+7.309%; +8.105%]	1.258s	1.168s
scenario:startup:insecure-bank:iast:Remote Config	better [-98.493µs; -75.956µs] or [-16.435%; -12.674%]	512.069µs	599.293µs
scenario:startup:insecure-bank:iast:Telemetry	worse [+3.445ms; +3.791ms] or [+40.175%; +44.204%]	12.194ms	8.576ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent	worse [+546.489ms; +776.585ms] or [+5.918%; +8.409%]	9.896s	9.235s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start	worse [+65.031ms; +108.302ms] or [+5.521%; +9.194%]	1.265s	1.178s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config	better [-131.959µs; -81.693µs] or [-21.318%; -13.198%]	512.169µs	618.995µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Telemetry	worse [+3.256ms; +3.731ms] or [+37.026%; +42.428%]	12.289ms	8.795ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent	worse [+530.620ms; +707.180ms] or [+5.718%; +7.620%]	9.899s	9.280s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start	worse [+78.002ms; +110.520ms] or [+6.692%; +9.482%]	1.260s	1.166s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry	worse [+3.267ms; +3.793ms] or [+38.362%; +44.540%]	12.046ms	8.516ms
scenario:startup:insecure-bank:tracing:Agent	worse [+555.218ms; +655.401ms] or [+6.404%; +7.560%]	9.275s	8.670s
scenario:startup:insecure-bank:tracing:Agent.start	worse [+77.018ms; +114.285ms] or [+7.359%; +10.920%]	1.142s	1.047s
scenario:startup:insecure-bank:tracing:Remote Config	better [-123.993µs; -84.183µs] or [-17.857%; -12.124%]	590.258µs	694.346µs
scenario:startup:petclinic:appsec:Agent	worse [+839.237ms; +1004.775ms] or [+7.802%; +9.341%]	11.679s	10.757s
scenario:startup:petclinic:appsec:Agent.start	worse [+81.446ms; +96.891ms] or [+6.894%; +8.202%]	1.271s	1.181s
scenario:startup:petclinic:appsec:Remote Config	better [-46.162µs; -17.568µs] or [-7.044%; -2.681%]	623.495µs	655.361µs
scenario:startup:petclinic:appsec:Telemetry	worse [+3.920ms; +4.204ms] or [+47.589%; +51.037%]	12.298ms	8.236ms
scenario:startup:petclinic:iast:Agent	worse [+794.882ms; +1042.601ms] or [+7.214%; +9.462%]	11.938s	11.019s
scenario:startup:petclinic:iast:Agent.start	worse [+81.997ms; +124.406ms] or [+7.009%; +10.634%]	1.273s	1.170s
scenario:startup:petclinic:iast:Telemetry	worse [+3.297ms; +3.811ms] or [+37.885%; +43.799%]	12.255ms	8.702ms
scenario:startup:petclinic:profiling:Agent	worse [+803.944ms; +1018.891ms] or [+7.394%; +9.371%]	11.784s	10.873s
scenario:startup:petclinic:profiling:Agent.start	worse [+113.960ms; +124.948ms] or [+9.052%; +9.924%]	1.378s	1.259s
scenario:startup:petclinic:profiling:ProfilingAgent	worse [+26.352ms; +28.571ms] or [+27.591%; +29.914%]	122.970ms	95.508ms
scenario:startup:petclinic:profiling:Remote Config	better [-65.756µs; -32.134µs] or [-9.808%; -4.793%]	621.506µs	670.452µs
scenario:startup:petclinic:profiling:Telemetry	worse [+3.393ms; +3.807ms] or [+37.839%; +42.446%]	12.568ms	8.968ms
scenario:startup:petclinic:profiling:Profiling	worse [+26.348ms; +28.566ms] or [+27.580%; +29.902%]	122.990ms	95.533ms
scenario:startup:petclinic:tracing:Agent	worse [+778.194ms; +1097.162ms] or [+7.414%; +10.453%]	11.433s	10.496s
scenario:startup:petclinic:tracing:Agent.start	worse [+80.022ms; +98.063ms] or [+7.663%; +9.391%]	1.133s	1.044s
scenario:startup:petclinic:tracing:Remote Config	better [-119.642µs; -74.353µs] or [-17.603%; -10.940%]	582.671µs	679.669µs

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046604
Total [baseline] (8.67 s) : 0, 8669857
Agent [candidate] (1.142 s) : 0, 1142256
Total [candidate] (9.275 s) : 0, 9275167
section iast
Agent [baseline] (1.168 s) : 0, 1167839
Total [baseline] (9.288 s) : 0, 9288426
Agent [candidate] (1.258 s) : 0, 1257838
Total [candidate] (9.872 s) : 0, 9872320
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.178 s) : 0, 1177932
Total [baseline] (9.235 s) : 0, 9234738
Agent [candidate] (1.265 s) : 0, 1264599
Total [candidate] (9.896 s) : 0, 9896275
section iast_TELEMETRY_OFF
Agent [baseline] (1.166 s) : 0, 1165552
Total [baseline] (9.28 s) : 0, 9280146
Agent [candidate] (1.26 s) : 0, 1259813
Total [candidate] (9.899 s) : 0, 9899046

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.168 s	121.234 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.178 s	131.328 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.166 s	118.947 ms (11.4%)
Total	tracing	8.67 s	-
Total	iast	9.288 s	618.569 ms (7.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.235 s	564.881 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.28 s	610.289 ms (7.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.142 s	-
Agent	iast	1.258 s	115.582 ms (10.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.265 s	122.343 ms (10.7%)
Agent	iast_TELEMETRY_OFF	1.26 s	117.557 ms (10.3%)
Total	tracing	9.275 s	-
Total	iast	9.872 s	597.153 ms (6.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.896 s	621.108 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.899 s	623.88 ms (6.7%)

gantt
title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (721.277 ms) : 0, 721277
BytebuddyAgent [candidate] (722.443 ms) : 0, 722443
GlobalTracer [baseline] (240.022 ms) : 0, 240022
GlobalTracer [candidate] (240.68 ms) : 0, 240680
AppSec [baseline] (55.748 ms) : 0, 55748
AppSec [candidate] (55.667 ms) : 0, 55667
Debugger [candidate] (95.342 ms) : 0, 95342
Remote Config [baseline] (694.346 µs) : 0, 694
Remote Config [candidate] (590.258 µs) : 0, 590
Telemetry [baseline] (13.813 ms) : 0, 13813
Telemetry [candidate] (12.421 ms) : 0, 12421
section iast
BytebuddyAgent [baseline] (834.875 ms) : 0, 834875
BytebuddyAgent [candidate] (836.688 ms) : 0, 836688
GlobalTracer [baseline] (229.553 ms) : 0, 229553
GlobalTracer [candidate] (230.141 ms) : 0, 230141
IAST [baseline] (22.468 ms) : 0, 22468
IAST [candidate] (22.859 ms) : 0, 22859
AppSec [baseline] (56.832 ms) : 0, 56832
AppSec [candidate] (57.176 ms) : 0, 57176
Debugger [candidate] (83.283 ms) : 0, 83283
Remote Config [baseline] (599.293 µs) : 0, 599
Remote Config [candidate] (512.069 µs) : 0, 512
Telemetry [baseline] (8.576 ms) : 0, 8576
Telemetry [candidate] (12.194 ms) : 0, 12194
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (841.749 ms) : 0, 841749
BytebuddyAgent [candidate] (841.117 ms) : 0, 841117
GlobalTracer [baseline] (230.913 ms) : 0, 230913
GlobalTracer [candidate] (231.414 ms) : 0, 231414
IAST [baseline] (23.07 ms) : 0, 23070
IAST [candidate] (23.085 ms) : 0, 23085
AppSec [baseline] (57.759 ms) : 0, 57759
AppSec [candidate] (57.535 ms) : 0, 57535
Debugger [candidate] (83.515 ms) : 0, 83515
Remote Config [baseline] (618.995 µs) : 0, 619
Remote Config [candidate] (512.169 µs) : 0, 512
Telemetry [baseline] (8.795 ms) : 0, 8795
Telemetry [candidate] (12.289 ms) : 0, 12289
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (833.074 ms) : 0, 833074
BytebuddyAgent [candidate] (837.92 ms) : 0, 837920
GlobalTracer [baseline] (229.592 ms) : 0, 229592
GlobalTracer [candidate] (230.583 ms) : 0, 230583
IAST [baseline] (24.196 ms) : 0, 24196
IAST [candidate] (23.308 ms) : 0, 23308
AppSec [baseline] (54.624 ms) : 0, 54624
AppSec [candidate] (56.653 ms) : 0, 56653
Debugger [candidate] (83.679 ms) : 0, 83679
Remote Config [baseline] (604.096 µs) : 0, 604
Remote Config [candidate] (537.468 µs) : 0, 537
Telemetry [baseline] (8.516 ms) : 0, 8516
Telemetry [candidate] (12.046 ms) : 0, 12046

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1044252
Total [baseline] (10.496 s) : 0, 10495770
Agent [candidate] (1.133 s) : 0, 1133295
Total [candidate] (11.433 s) : 0, 11433448
section appsec
Agent [baseline] (1.181 s) : 0, 1181375
Total [baseline] (10.757 s) : 0, 10756585
Agent [candidate] (1.271 s) : 0, 1270544
Total [candidate] (11.679 s) : 0, 11678591
section iast
Agent [baseline] (1.17 s) : 0, 1169884
Total [baseline] (11.019 s) : 0, 11018807
Agent [candidate] (1.273 s) : 0, 1273086
Total [candidate] (11.938 s) : 0, 11937549
section profiling
Agent [baseline] (1.259 s) : 0, 1259016
Total [baseline] (10.873 s) : 0, 10873021
Agent [candidate] (1.378 s) : 0, 1378470
Total [candidate] (11.784 s) : 0, 11784439

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	appsec	1.181 s	137.123 ms (13.1%)
Agent	iast	1.17 s	125.632 ms (12.0%)
Agent	profiling	1.259 s	214.764 ms (20.6%)
Total	tracing	10.496 s	-
Total	appsec	10.757 s	260.815 ms (2.5%)
Total	iast	11.019 s	523.037 ms (5.0%)
Total	profiling	10.873 s	377.251 ms (3.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.133 s	-
Agent	appsec	1.271 s	137.249 ms (12.1%)
Agent	iast	1.273 s	139.791 ms (12.3%)
Agent	profiling	1.378 s	245.175 ms (21.6%)
Total	tracing	11.433 s	-
Total	appsec	11.679 s	245.143 ms (2.1%)
Total	iast	11.938 s	504.101 ms (4.4%)
Total	profiling	11.784 s	350.99 ms (3.1%)

gantt
title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.501 ms) : 0, 720501
BytebuddyAgent [candidate] (716.459 ms) : 0, 716459
GlobalTracer [baseline] (238.877 ms) : 0, 238877
GlobalTracer [candidate] (238.598 ms) : 0, 238598
AppSec [baseline] (55.552 ms) : 0, 55552
AppSec [candidate] (55.377 ms) : 0, 55377
Debugger [candidate] (94.821 ms) : 0, 94821
Remote Config [baseline] (679.669 µs) : 0, 680
Remote Config [candidate] (582.671 µs) : 0, 583
Telemetry [baseline] (13.548 ms) : 0, 13548
Telemetry [candidate] (12.464 ms) : 0, 12464
section appsec
BytebuddyAgent [baseline] (734.017 ms) : 0, 734017
BytebuddyAgent [candidate] (734.888 ms) : 0, 734888
GlobalTracer [baseline] (235.636 ms) : 0, 235636
GlobalTracer [candidate] (235.669 ms) : 0, 235669
AppSec [baseline] (177.167 ms) : 0, 177167
AppSec [candidate] (177.355 ms) : 0, 177355
Debugger [candidate] (84.008 ms) : 0, 84008
Remote Config [baseline] (655.361 µs) : 0, 655
Remote Config [candidate] (623.495 µs) : 0, 623
Telemetry [baseline] (8.236 ms) : 0, 8236
Telemetry [candidate] (12.298 ms) : 0, 12298
IAST [baseline] (21.521 ms) : 0, 21521
IAST [candidate] (21.329 ms) : 0, 21329
section iast
BytebuddyAgent [baseline] (835.833 ms) : 0, 835833
BytebuddyAgent [candidate] (848.442 ms) : 0, 848442
GlobalTracer [baseline] (229.828 ms) : 0, 229828
GlobalTracer [candidate] (232.039 ms) : 0, 232039
AppSec [baseline] (57.055 ms) : 0, 57055
AppSec [candidate] (57.584 ms) : 0, 57584
Debugger [candidate] (83.951 ms) : 0, 83951
Remote Config [baseline] (618.506 µs) : 0, 619
Remote Config [candidate] (519.185 µs) : 0, 519
Telemetry [baseline] (8.702 ms) : 0, 8702
Telemetry [candidate] (12.255 ms) : 0, 12255
IAST [baseline] (22.826 ms) : 0, 22826
IAST [candidate] (23.051 ms) : 0, 23051
section profiling
ProfilingAgent [baseline] (95.508 ms) : 0, 95508
ProfilingAgent [candidate] (122.97 ms) : 0, 122970
BytebuddyAgent [baseline] (708.779 ms) : 0, 708779
BytebuddyAgent [candidate] (709.016 ms) : 0, 709016
GlobalTracer [baseline] (349.472 ms) : 0, 349472
GlobalTracer [candidate] (350.438 ms) : 0, 350438
AppSec [baseline] (55.297 ms) : 0, 55297
AppSec [candidate] (54.602 ms) : 0, 54602
Debugger [candidate] (87.889 ms) : 0, 87889
Remote Config [baseline] (670.452 µs) : 0, 670
Remote Config [candidate] (621.506 µs) : 0, 622
Telemetry [baseline] (8.968 ms) : 0, 8968
Telemetry [candidate] (12.568 ms) : 0, 12568
Profiling [baseline] (95.533 ms) : 0, 95533
Profiling [candidate] (122.99 ms) : 0, 122990

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-02-28T16:17:47	2025-02-28T16:25:32
git_branch	master	jpbempel/in-product-enablement
git_commit_date	1740758111	1740758874
git_commit_sha	`19d8832`	`676ddbb`
release_version	1.47.0-SNAPSHOT~19d8832a9d	1.47.0-SNAPSHOT~676ddbb84a
start_time	2025-02-28T16:17:33	2025-02-28T16:25:18

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1740760338	1740760338
ci_job_id	828044653	828044653
ci_pipeline_id	57360274	57360274
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-v1phgxi-project-304-concurrent-0-5s0wlkm5 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-v1phgxi-project-304-concurrent-0-5s0wlkm5 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section baseline
no_agent (379.024 µs) : 359, 399
. : milestone, 379,
iast (515.275 µs) : 493, 537
. : milestone, 515,
iast_FULL (730.761 µs) : 709, 753
. : milestone, 731,
iast_GLOBAL (558.021 µs) : 536, 580
. : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (509.59 µs) : 487, 532
. : milestone, 510,
iast_INACTIVE (456.306 µs) : 435, 477
. : milestone, 456,
iast_TELEMETRY_OFF (495.465 µs) : 472, 519
. : milestone, 495,
tracing (453.309 µs) : 433, 474
. : milestone, 453,
section candidate
no_agent (381.435 µs) : 361, 402
. : milestone, 381,
iast (517.992 µs) : 496, 540
. : milestone, 518,
iast_FULL (731.44 µs) : 709, 754
. : milestone, 731,
iast_GLOBAL (559.002 µs) : 536, 582
. : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (511.658 µs) : 490, 533
. : milestone, 512,
iast_INACTIVE (465.307 µs) : 443, 487
. : milestone, 465,
iast_TELEMETRY_OFF (500.704 µs) : 477, 524
. : milestone, 501,
tracing (454.323 µs) : 434, 475
. : milestone, 454,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	379.024 µs [359.417 µs, 398.63 µs]	-
iast	515.275 µs [493.343 µs, 537.206 µs]	136.251 µs (35.9%)
iast_FULL	730.761 µs [708.769 µs, 752.753 µs]	351.737 µs (92.8%)
iast_GLOBAL	558.021 µs [535.624 µs, 580.418 µs]	178.997 µs (47.2%)
iast_HARDCODED_SECRET_DISABLED	509.59 µs [487.474 µs, 531.705 µs]	130.566 µs (34.4%)
iast_INACTIVE	456.306 µs [435.384 µs, 477.228 µs]	77.282 µs (20.4%)
iast_TELEMETRY_OFF	495.465 µs [472.09 µs, 518.839 µs]	116.441 µs (30.7%)
tracing	453.309 µs [432.612 µs, 474.007 µs]	74.286 µs (19.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.435 µs [360.747 µs, 402.123 µs]	-
iast	517.992 µs [495.807 µs, 540.178 µs]	136.557 µs (35.8%)
iast_FULL	731.44 µs [708.737 µs, 754.143 µs]	350.006 µs (91.8%)
iast_GLOBAL	559.002 µs [536.389 µs, 581.615 µs]	177.567 µs (46.6%)
iast_HARDCODED_SECRET_DISABLED	511.658 µs [489.986 µs, 533.33 µs]	130.223 µs (34.1%)
iast_INACTIVE	465.307 µs [443.437 µs, 487.177 µs]	83.872 µs (22.0%)
iast_TELEMETRY_OFF	500.704 µs [477.226 µs, 524.181 µs]	119.269 µs (31.3%)
tracing	454.323 µs [433.578 µs, 475.068 µs]	72.888 µs (19.1%)

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section baseline
no_agent (1.363 ms) : 1343, 1383
. : milestone, 1363,
appsec (1.733 ms) : 1709, 1756
. : milestone, 1733,
appsec_no_iast (1.745 ms) : 1721, 1769
. : milestone, 1745,
code_origins (1.693 ms) : 1660, 1727
. : milestone, 1693,
iast (1.489 ms) : 1465, 1512
. : milestone, 1489,
profiling (1.554 ms) : 1530, 1578
. : milestone, 1554,
tracing (1.489 ms) : 1463, 1514
. : milestone, 1489,
section candidate
no_agent (1.368 ms) : 1349, 1387
. : milestone, 1368,
appsec (1.715 ms) : 1690, 1739
. : milestone, 1715,
appsec_no_iast (1.716 ms) : 1692, 1739
. : milestone, 1716,
code_origins (1.689 ms) : 1656, 1722
. : milestone, 1689,
iast (1.519 ms) : 1495, 1544
. : milestone, 1519,
profiling (1.503 ms) : 1478, 1528
. : milestone, 1503,
tracing (1.482 ms) : 1457, 1506
. : milestone, 1482,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.363 ms [1.343 ms, 1.383 ms]	-
appsec	1.733 ms [1.709 ms, 1.756 ms]	369.182 µs (27.1%)
appsec_no_iast	1.745 ms [1.721 ms, 1.769 ms]	381.354 µs (28.0%)
code_origins	1.693 ms [1.66 ms, 1.727 ms]	329.91 µs (24.2%)
iast	1.489 ms [1.465 ms, 1.512 ms]	125.475 µs (9.2%)
profiling	1.554 ms [1.53 ms, 1.578 ms]	190.439 µs (14.0%)
tracing	1.489 ms [1.463 ms, 1.514 ms]	125.379 µs (9.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.368 ms [1.349 ms, 1.387 ms]	-
appsec	1.715 ms [1.69 ms, 1.739 ms]	346.906 µs (25.4%)
appsec_no_iast	1.716 ms [1.692 ms, 1.739 ms]	347.968 µs (25.4%)
code_origins	1.689 ms [1.656 ms, 1.722 ms]	321.15 µs (23.5%)
iast	1.519 ms [1.495 ms, 1.544 ms]	151.772 µs (11.1%)
profiling	1.503 ms [1.478 ms, 1.528 ms]	135.601 µs (9.9%)
tracing	1.482 ms [1.457 ms, 1.506 ms]	113.851 µs (8.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/in-product-enablement
git_commit_date	1740758111	1740758874
git_commit_sha	`19d8832`	`676ddbb`
release_version	1.47.0-SNAPSHOT~19d8832a9d	1.47.0-SNAPSHOT~676ddbb84a

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1740760861	1740760861
ci_job_id	828044654	828044654
ci_pipeline_id	57360274	57360274
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8g9zkp7x-project-304-concurrent-1-gkj7rlmw 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8g9zkp7x-project-304-concurrent-1-gkj7rlmw 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (2.332 ms) : 2288, 2375
. : milestone, 2332,
iast (2.121 ms) : 2066, 2176
. : milestone, 2121,
iast_GLOBAL (2.153 ms) : 2098, 2209
. : milestone, 2153,
profiling (1.968 ms) : 1925, 2011
. : milestone, 1968,
tracing (1.954 ms) : 1912, 1997
. : milestone, 1954,
section candidate
no_agent (1.471 ms) : 1460, 1483
. : milestone, 1471,
appsec (2.338 ms) : 2294, 2381
. : milestone, 2338,
iast (2.12 ms) : 2064, 2176
. : milestone, 2120,
iast_GLOBAL (2.159 ms) : 2103, 2214
. : milestone, 2159,
profiling (1.969 ms) : 1926, 2013
. : milestone, 1969,
tracing (1.943 ms) : 1901, 1986
. : milestone, 1943,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.465 ms, 1.488 ms]	-
appsec	2.332 ms [2.288 ms, 2.375 ms]	855.193 µs (57.9%)
iast	2.121 ms [2.066 ms, 2.176 ms]	644.861 µs (43.7%)
iast_GLOBAL	2.153 ms [2.098 ms, 2.209 ms]	676.912 µs (45.9%)
profiling	1.968 ms [1.925 ms, 2.011 ms]	491.736 µs (33.3%)
tracing	1.954 ms [1.912 ms, 1.997 ms]	477.878 µs (32.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	2.338 ms [2.294 ms, 2.381 ms]	866.564 µs (58.9%)
iast	2.12 ms [2.064 ms, 2.176 ms]	648.876 µs (44.1%)
iast_GLOBAL	2.159 ms [2.103 ms, 2.214 ms]	687.732 µs (46.7%)
profiling	1.969 ms [1.926 ms, 2.013 ms]	498.321 µs (33.9%)
tracing	1.943 ms [1.901 ms, 1.986 ms]	472.307 µs (32.1%)

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~676ddbb84a, baseline=1.47.0-SNAPSHOT~19d8832a9d
dateFormat X
axisFormat %s
section baseline
no_agent (15.344 s) : 15344000, 15344000
. : milestone, 15344000,
appsec (15.141 s) : 15141000, 15141000
. : milestone, 15141000,
iast (18.265 s) : 18265000, 18265000
. : milestone, 18265000,
iast_GLOBAL (18.062 s) : 18062000, 18062000
. : milestone, 18062000,
profiling (14.999 s) : 14999000, 14999000
. : milestone, 14999000,
tracing (14.934 s) : 14934000, 14934000
. : milestone, 14934000,
section candidate
no_agent (15.729 s) : 15729000, 15729000
. : milestone, 15729000,
appsec (14.878 s) : 14878000, 14878000
. : milestone, 14878000,
iast (18.862 s) : 18862000, 18862000
. : milestone, 18862000,
iast_GLOBAL (18.154 s) : 18154000, 18154000
. : milestone, 18154000,
profiling (14.986 s) : 14986000, 14986000
. : milestone, 14986000,
tracing (14.867 s) : 14867000, 14867000
. : milestone, 14867000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.344 s [15.344 s, 15.344 s]	-
appsec	15.141 s [15.141 s, 15.141 s]	-203.0 ms (-1.3%)
iast	18.265 s [18.265 s, 18.265 s]	2.921 s (19.0%)
iast_GLOBAL	18.062 s [18.062 s, 18.062 s]	2.718 s (17.7%)
profiling	14.999 s [14.999 s, 14.999 s]	-345.0 ms (-2.2%)
tracing	14.934 s [14.934 s, 14.934 s]	-410.0 ms (-2.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.729 s [15.729 s, 15.729 s]	-
appsec	14.878 s [14.878 s, 14.878 s]	-851.0 ms (-5.4%)
iast	18.862 s [18.862 s, 18.862 s]	3.133 s (19.9%)
iast_GLOBAL	18.154 s [18.154 s, 18.154 s]	2.425 s (15.4%)
profiling	14.986 s [14.986 s, 14.986 s]	-743.0 ms (-4.7%)
tracing	14.867 s [14.867 s, 14.867 s]	-862.0 ms (-5.5%)

evanchooly

a few questions but looks great overall.

evanchooly · 2025-02-27T15:09:48Z

can we also rename this to codeOrigin... while we're renaming things?

evanchooly · 2025-02-27T15:10:28Z

do we still want to bind code origin to DI enablement?

it's not bond.

evanchooly · 2025-02-27T15:22:38Z

StringJoiner might be easier to read here.

evanchooly · 2025-02-27T15:23:13Z

maybe rename the method, too? or is this name tied to serialization now?

Introduce the ability to start and stop Debugger features: - Dynamic Instrumentation - Exception Replay - Code Origin - Distributed Debugger dynamically based on RemoteConfig record: APM_TRACING DebuggerAgent is now run every time at startup to have the base of some feature ready and be able to start the minimum required foe each feature. Ability to stop also the feature at any time to uninstall probes. Add smoke tests

This reverts commit dceca28.

jpbempel requested a review from evanchooly February 27, 2025 14:46

evanchooly approved these changes Feb 27, 2025
View reviewed changes

jpbempel added 4 commits February 28, 2025 17:07

fix tests
81a8048

address comments
e9046b2

Update system-tests to 6980534f333b3f7a35d83df2230f00f4e26642f5
676ddbb

jpbempel marked this pull request as ready for review February 28, 2025 16:08

jpbempel force-pushed the jpbempel/in-product-enablement branch from 2a5edcf to 676ddbbCompare February 28, 2025 16:08

jpbempel requested review from a team as code owners February 28, 2025 16:08

jpbempel requested review from dougqh and ojung and removed request for a teamFebruary 28, 2025 16:08

jpbempel merged commit dceca28 into masterFeb 28, 2025

jpbempel deleted the jpbempel/in-product-enablement branch February 28, 2025 17:56

github-actionsBot added this to the 1.47.0 milestone Feb 28, 2025

jpbempel added comp: debugger Dynamic Instrumentation type: feature Enhancements and improvements labels Mar 3, 2025

nikita-tkachenko-datadog mentioned this pull request Mar 4, 2025
Disable debugger by default if CI Visibility is enabled #8506
Merged

jpbempel restored the jpbempel/in-product-enablement branch March 4, 2025 20:28

jpbempel added a commit that referenced this pull request Mar 4, 2025
Revert "Add In-Product Enablement (#8461)"
d1c1e75
This reverts commit dceca28.

jpbempel added a commit that referenced this pull request Mar 4, 2025
Revert "Add In-Product Enablement (#8461)" (#8509)
7787c9d
This reverts commit dceca28.

mcculls added the tag: no release notes Changes to exclude from release notes label Mar 4, 2025

Navigation Menu

Uh oh!

Add In-Product Enablement - #8461

Add In-Product Enablement#8461
jpbempel merged 4 commits into
masterfrom
jpbempel/in-product-enablement

jpbempel commented Feb 27, 2025•
edited by atlassian Bot
Loading

Uh oh!

pr-commenterBot commented Feb 27, 2025•
edited
Loading

Uh oh!

pr-commenterBot commented Feb 27, 2025•
edited
Loading

Uh oh!

evanchooly left a comment

Uh oh!

evanchoolyFeb 27, 2025

Uh oh!

jpbempelFeb 27, 2025

Uh oh!

evanchoolyFeb 27, 2025

Uh oh!

jpbempelFeb 27, 2025

Uh oh!

evanchoolyFeb 27, 2025

Uh oh!

jpbempelFeb 27, 2025

Uh oh!

evanchoolyFeb 27, 2025

Uh oh!

jpbempelFeb 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jpbempel commented Feb 27, 2025• edited by atlassian BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Feb 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

pr-commenterBot commented Feb 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

evanchooly left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jpbempel commented Feb 27, 2025•
edited by atlassian Bot
Loading

pr-commenterBot commented Feb 27, 2025•
edited
Loading

pr-commenterBot commented Feb 27, 2025•
edited
Loading