Add telemetry for the RUM injector by sarahchen6 · Pull Request #9267 · DataDog/dd-trace-java

sarahchen6 · 2025-07-28T21:10:33Z

What Does This Do

Add telemetry for the RUM injector. Gather injection metrics and tags, and send the data via StatsDClient. The common metrics are based on: https://github.com/DataDog/dd-go/blob/prod/trace/apps/tracer-telemetry-intake/telemetry-metrics/static/common_metrics.json

Motivation

Complete the RUM Injector requirements (doc)

Additional Notes

This telemetry implementation does not include the following metrics:

injection.initialization.failed: Initialization failures can be due to a multitude of reasons and are already covered by SSI telemetry
injection.installation: Installation metrics are related to web server installers and not captured by this RUM injector implementation
injection.installation.duration: Installation metrics are related to web server installers and not captured by this RUM injector implementation

It also does not include the injector_version tag for the injection.failed metric. This tag refers to the Rust injector and we do not implement that.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/LANGPLAT-679

pr-commenter · 2025-07-28T21:48:07Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/implement-rum-injector-telemetry
git_commit_date	1755621799	1755631510
git_commit_sha	`27cd7f4`	`e9095b5`
release_version	1.53.0-SNAPSHOT~27cd7f43d9	1.53.0-SNAPSHOT~e9095b558b

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1755633311	1755633311
ci_job_id	1088151223	1088151223
ci_pipeline_id	74169254	74169254
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ns7pu4gu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ns7pu4gu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1051571
Total [baseline] (8.668 s) : 0, 8667699
Agent [candidate] (1.051 s) : 0, 1051243
Total [candidate] (8.652 s) : 0, 8651882
section iast
Agent [baseline] (1.201 s) : 0, 1200998
Total [baseline] (9.395 s) : 0, 9394589
Agent [candidate] (1.182 s) : 0, 1182393
Total [candidate] (9.336 s) : 0, 9335893

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.201 s	149.427 ms (14.2%)
Total	tracing	8.668 s	-
Total	iast	9.395 s	726.891 ms (8.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	iast	1.182 s	131.15 ms (12.5%)
Total	tracing	8.652 s	-
Total	iast	9.336 s	684.011 ms (7.9%)

gantt
title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.471 ms) : 0, 1471
BytebuddyAgent [baseline] (736.502 ms) : 0, 736502
BytebuddyAgent [candidate] (736.823 ms) : 0, 736823
GlobalTracer [baseline] (243.819 ms) : 0, 243819
GlobalTracer [candidate] (242.808 ms) : 0, 242808
AppSec [baseline] (30.271 ms) : 0, 30271
AppSec [candidate] (30.032 ms) : 0, 30032
Debugger [baseline] (6.11 ms) : 0, 6110
Debugger [candidate] (6.03 ms) : 0, 6030
Remote Config [baseline] (672.375 µs) : 0, 672
Remote Config [candidate] (660.973 µs) : 0, 661
Telemetry [baseline] (10.912 ms) : 0, 10912
Telemetry [candidate] (12.276 ms) : 0, 12276
section iast
crashtracking [baseline] (1.492 ms) : 0, 1492
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (868.458 ms) : 0, 868458
BytebuddyAgent [candidate] (853.106 ms) : 0, 853106
GlobalTracer [baseline] (236.069 ms) : 0, 236069
GlobalTracer [candidate] (233.957 ms) : 0, 233957
AppSec [baseline] (27.895 ms) : 0, 27895
AppSec [candidate] (27.834 ms) : 0, 27834
Debugger [baseline] (6.664 ms) : 0, 6664
Debugger [candidate] (5.799 ms) : 0, 5799
Remote Config [baseline] (606.786 µs) : 0, 607
Remote Config [candidate] (617.742 µs) : 0, 618
Telemetry [baseline] (8.463 ms) : 0, 8463
Telemetry [candidate] (8.489 ms) : 0, 8489
IAST [baseline] (30.024 ms) : 0, 30024
IAST [candidate] (30.094 ms) : 0, 30094

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1049804
Total [baseline] (10.706 s) : 0, 10706013
Agent [candidate] (1.047 s) : 0, 1047239
Total [candidate] (10.791 s) : 0, 10790805
section appsec
Agent [baseline] (1.231 s) : 0, 1231180
Total [baseline] (10.9 s) : 0, 10899853
Agent [candidate] (1.233 s) : 0, 1233491
Total [candidate] (10.868 s) : 0, 10868119
section iast
Agent [baseline] (1.192 s) : 0, 1192214
Total [baseline] (10.956 s) : 0, 10956312
Agent [candidate] (1.191 s) : 0, 1191286
Total [candidate] (10.985 s) : 0, 10985124
section profiling
Agent [baseline] (1.21 s) : 0, 1209699
Total [baseline] (11.134 s) : 0, 11133911
Agent [candidate] (1.197 s) : 0, 1196764
Total [candidate] (10.977 s) : 0, 10977017

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.05 s	-
Agent	appsec	1.231 s	181.376 ms (17.3%)
Agent	iast	1.192 s	142.41 ms (13.6%)
Agent	profiling	1.21 s	159.895 ms (15.2%)
Total	tracing	10.706 s	-
Total	appsec	10.9 s	193.84 ms (1.8%)
Total	iast	10.956 s	250.299 ms (2.3%)
Total	profiling	11.134 s	427.898 ms (4.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	appsec	1.233 s	186.253 ms (17.8%)
Agent	iast	1.191 s	144.047 ms (13.8%)
Agent	profiling	1.197 s	149.525 ms (14.3%)
Total	tracing	10.791 s	-
Total	appsec	10.868 s	77.314 ms (0.7%)
Total	iast	10.985 s	194.319 ms (1.8%)
Total	profiling	10.977 s	186.213 ms (1.7%)

gantt
title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.446 ms) : 0, 1446
BytebuddyAgent [baseline] (733.924 ms) : 0, 733924
BytebuddyAgent [candidate] (732.535 ms) : 0, 732535
GlobalTracer [baseline] (243.283 ms) : 0, 243283
GlobalTracer [candidate] (242.955 ms) : 0, 242955
AppSec [baseline] (30.107 ms) : 0, 30107
AppSec [candidate] (30.206 ms) : 0, 30206
Debugger [baseline] (6.086 ms) : 0, 6086
Debugger [candidate] (6.124 ms) : 0, 6124
Remote Config [baseline] (698.303 µs) : 0, 698
Remote Config [candidate] (662.761 µs) : 0, 663
Telemetry [baseline] (13.039 ms) : 0, 13039
Telemetry [candidate] (12.239 ms) : 0, 12239
section appsec
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (760.865 ms) : 0, 760865
BytebuddyAgent [candidate] (762.453 ms) : 0, 762453
GlobalTracer [baseline] (237.166 ms) : 0, 237166
GlobalTracer [candidate] (237.752 ms) : 0, 237752
IAST [baseline] (23.951 ms) : 0, 23951
IAST [candidate] (23.846 ms) : 0, 23846
AppSec [baseline] (171.64 ms) : 0, 171640
AppSec [candidate] (169.655 ms) : 0, 169655
Debugger [baseline] (5.725 ms) : 0, 5725
Debugger [candidate] (7.208 ms) : 0, 7208
Remote Config [baseline] (628.851 µs) : 0, 629
Remote Config [candidate] (629.176 µs) : 0, 629
Telemetry [baseline] (8.429 ms) : 0, 8429
Telemetry [candidate] (9.192 ms) : 0, 9192
section iast
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.464 ms) : 0, 1464
BytebuddyAgent [baseline] (860.967 ms) : 0, 860967
BytebuddyAgent [candidate] (859.95 ms) : 0, 859950
GlobalTracer [baseline] (235.212 ms) : 0, 235212
GlobalTracer [candidate] (235.233 ms) : 0, 235233
IAST [baseline] (29.613 ms) : 0, 29613
IAST [candidate] (29.965 ms) : 0, 29965
AppSec [baseline] (27.861 ms) : 0, 27861
AppSec [candidate] (27.744 ms) : 0, 27744
Debugger [baseline] (6.69 ms) : 0, 6690
Debugger [candidate] (6.633 ms) : 0, 6633
Remote Config [baseline] (615.418 µs) : 0, 615
Remote Config [candidate] (623.917 µs) : 0, 624
Telemetry [baseline] (8.502 ms) : 0, 8502
Telemetry [candidate] (8.426 ms) : 0, 8426
section profiling
crashtracking [baseline] (1.434 ms) : 0, 1434
crashtracking [candidate] (1.433 ms) : 0, 1433
BytebuddyAgent [baseline] (770.045 ms) : 0, 770045
BytebuddyAgent [candidate] (761.677 ms) : 0, 761677
GlobalTracer [baseline] (224.323 ms) : 0, 224323
GlobalTracer [candidate] (222.638 ms) : 0, 222638
AppSec [baseline] (30.375 ms) : 0, 30375
AppSec [candidate] (30.134 ms) : 0, 30134
Debugger [baseline] (6.32 ms) : 0, 6320
Debugger [candidate] (6.302 ms) : 0, 6302
Remote Config [baseline] (726.212 µs) : 0, 726
Remote Config [candidate] (727.627 µs) : 0, 728
Telemetry [baseline] (15.814 ms) : 0, 15814
Telemetry [candidate] (16.273 ms) : 0, 16273
ProfilingAgent [baseline] (110.585 ms) : 0, 110585
ProfilingAgent [candidate] (107.984 ms) : 0, 107984
Profiling [baseline] (111.255 ms) : 0, 111255
Profiling [candidate] (108.625 ms) : 0, 108625

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/implement-rum-injector-telemetry
git_commit_date	1755621799	1755631510
git_commit_sha	`27cd7f4`	`e9095b5`
release_version	1.53.0-SNAPSHOT~27cd7f43d9	1.53.0-SNAPSHOT~e9095b558b

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1755633093	1755633093
ci_job_id	1088151224	1088151224
ci_pipeline_id	74169254	74169254
cpu_model	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-tb41jv1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-tb41jv1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:tracing:high_load	worse [+261.174µs; +525.754µs] or [+3.301%; +6.645%]	unstable [-96.489op/s; +41.177op/s] or [-16.475%; +7.031%]	8.305ms	558.000op/s	7.912ms	585.656op/s
scenario:load:petclinic:no_agent:high_load	better [-1.485ms; -0.814ms] or [-3.807%; -2.087%]	unstable [-6.127op/s; +10.215op/s] or [-5.044%; +8.410%]	37.867ms	123.513op/s	39.016ms	121.468op/s
scenario:load:petclinic:appsec:high_load	better [-2.116ms; -1.187ms] or [-4.313%; -2.418%]	unstable [-3.865op/s; +10.540op/s] or [-4.052%; +11.050%]	47.413ms	98.725op/s	49.065ms	95.388op/s
scenario:load:petclinic:iast:high_load	worse [+1.125ms; +2.002ms] or [+2.442%; +4.345%]	unstable [-10.515op/s; +3.890op/s] or [-10.353%; +3.830%]	47.647ms	98.250op/s	46.083ms	101.562op/s

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section baseline
no_agent (39.016 ms) : 38693, 39340
. : milestone, 39016,
appsec (49.065 ms) : 48620, 49509
. : milestone, 49065,
code_origins (46.527 ms) : 46125, 46929
. : milestone, 46527,
iast (46.083 ms) : 45685, 46482
. : milestone, 46083,
profiling (50.047 ms) : 49598, 50496
. : milestone, 50047,
tracing (45.073 ms) : 44681, 45465
. : milestone, 45073,
section candidate
no_agent (37.867 ms) : 37567, 38166
. : milestone, 37867,
appsec (47.413 ms) : 46994, 47832
. : milestone, 47413,
code_origins (46.323 ms) : 45942, 46705
. : milestone, 46323,
iast (47.647 ms) : 47231, 48063
. : milestone, 47647,
profiling (49.301 ms) : 48813, 49789
. : milestone, 49301,
tracing (45.974 ms) : 45570, 46379
. : milestone, 45974,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	39.016 ms [38.693 ms, 39.34 ms]	-
appsec	49.065 ms [48.62 ms, 49.509 ms]	10.048 ms (25.8%)
code_origins	46.527 ms [46.125 ms, 46.929 ms]	7.51 ms (19.2%)
iast	46.083 ms [45.685 ms, 46.482 ms]	7.067 ms (18.1%)
profiling	50.047 ms [49.598 ms, 50.496 ms]	11.031 ms (28.3%)
tracing	45.073 ms [44.681 ms, 45.465 ms]	6.056 ms (15.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.867 ms [37.567 ms, 38.166 ms]	-
appsec	47.413 ms [46.994 ms, 47.832 ms]	9.547 ms (25.2%)
code_origins	46.323 ms [45.942 ms, 46.705 ms]	8.457 ms (22.3%)
iast	47.647 ms [47.231 ms, 48.063 ms]	9.78 ms (25.8%)
profiling	49.301 ms [48.813 ms, 49.789 ms]	11.434 ms (30.2%)
tracing	45.974 ms [45.57 ms, 46.379 ms]	8.108 ms (21.4%)

Request duration reports for insecure-bank

gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section baseline
no_agent (4.68 ms) : 4621, 4738
. : milestone, 4680,
iast (9.987 ms) : 9815, 10159
. : milestone, 9987,
iast_FULL (15.357 ms) : 15041, 15673
. : milestone, 15357,
iast_GLOBAL (11.145 ms) : 10943, 11347
. : milestone, 11145,
profiling (9.527 ms) : 9383, 9671
. : milestone, 9527,
tracing (7.912 ms) : 7793, 8030
. : milestone, 7912,
section candidate
no_agent (4.673 ms) : 4619, 4727
. : milestone, 4673,
iast (9.691 ms) : 9534, 9848
. : milestone, 9691,
iast_FULL (15.004 ms) : 14706, 15301
. : milestone, 15004,
iast_GLOBAL (11.24 ms) : 11022, 11459
. : milestone, 11240,
profiling (9.341 ms) : 9197, 9486
. : milestone, 9341,
tracing (8.305 ms) : 8177, 8433
. : milestone, 8305,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.68 ms [4.621 ms, 4.738 ms]	-
iast	9.987 ms [9.815 ms, 10.159 ms]	5.307 ms (113.4%)
iast_FULL	15.357 ms [15.041 ms, 15.673 ms]	10.677 ms (228.2%)
iast_GLOBAL	11.145 ms [10.943 ms, 11.347 ms]	6.466 ms (138.2%)
profiling	9.527 ms [9.383 ms, 9.671 ms]	4.847 ms (103.6%)
tracing	7.912 ms [7.793 ms, 8.03 ms]	3.232 ms (69.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.673 ms [4.619 ms, 4.727 ms]	-
iast	9.691 ms [9.534 ms, 9.848 ms]	5.018 ms (107.4%)
iast_FULL	15.004 ms [14.706 ms, 15.301 ms]	10.33 ms (221.1%)
iast_GLOBAL	11.24 ms [11.022 ms, 11.459 ms]	6.567 ms (140.5%)
profiling	9.341 ms [9.197 ms, 9.486 ms]	4.668 ms (99.9%)
tracing	8.305 ms [8.177 ms, 8.433 ms]	3.632 ms (77.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/implement-rum-injector-telemetry
git_commit_date	1755621799	1755631510
git_commit_sha	`27cd7f4`	`e9095b5`
release_version	1.53.0-SNAPSHOT~27cd7f43d9	1.53.0-SNAPSHOT~e9095b558b

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1755633556	1755633556
ci_job_id	1088151225	1088151225
ci_pipeline_id	74169254	74169254
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-azebvz3f 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-azebvz3f 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
. : milestone, 1480,
appsec (3.659 ms) : 3441, 3876
. : milestone, 3659,
iast (2.21 ms) : 2147, 2272
. : milestone, 2210,
iast_GLOBAL (2.254 ms) : 2191, 2317
. : milestone, 2254,
profiling (2.072 ms) : 2021, 2124
. : milestone, 2072,
tracing (2.017 ms) : 1969, 2065
. : milestone, 2017,
section candidate
no_agent (1.48 ms) : 1468, 1492
. : milestone, 1480,
appsec (3.604 ms) : 3389, 3818
. : milestone, 3604,
iast (2.213 ms) : 2150, 2277
. : milestone, 2213,
iast_GLOBAL (2.251 ms) : 2188, 2314
. : milestone, 2251,
profiling (2.056 ms) : 2004, 2108
. : milestone, 2056,
tracing (2.035 ms) : 1986, 2084
. : milestone, 2035,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	3.659 ms [3.441 ms, 3.876 ms]	2.178 ms (147.1%)
iast	2.21 ms [2.147 ms, 2.272 ms]	729.184 µs (49.3%)
iast_GLOBAL	2.254 ms [2.191 ms, 2.317 ms]	773.609 µs (52.3%)
profiling	2.072 ms [2.021 ms, 2.124 ms]	591.975 µs (40.0%)
tracing	2.017 ms [1.969 ms, 2.065 ms]	536.705 µs (36.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.468 ms, 1.492 ms]	-
appsec	3.604 ms [3.389 ms, 3.818 ms]	2.124 ms (143.5%)
iast	2.213 ms [2.15 ms, 2.277 ms]	733.504 µs (49.6%)
iast_GLOBAL	2.251 ms [2.188 ms, 2.314 ms]	771.259 µs (52.1%)
profiling	2.056 ms [2.004 ms, 2.108 ms]	576.381 µs (38.9%)
tracing	2.035 ms [1.986 ms, 2.084 ms]	555.003 µs (37.5%)

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~e9095b558b, baseline=1.53.0-SNAPSHOT~27cd7f43d9
dateFormat X
axisFormat %s
section baseline
no_agent (15.507 s) : 15507000, 15507000
. : milestone, 15507000,
appsec (14.911 s) : 14911000, 14911000
. : milestone, 14911000,
iast (18.67 s) : 18670000, 18670000
. : milestone, 18670000,
iast_GLOBAL (18.245 s) : 18245000, 18245000
. : milestone, 18245000,
profiling (15.131 s) : 15131000, 15131000
. : milestone, 15131000,
tracing (14.869 s) : 14869000, 14869000
. : milestone, 14869000,
section candidate
no_agent (15.494 s) : 15494000, 15494000
. : milestone, 15494000,
appsec (14.821 s) : 14821000, 14821000
. : milestone, 14821000,
iast (18.781 s) : 18781000, 18781000
. : milestone, 18781000,
iast_GLOBAL (18.108 s) : 18108000, 18108000
. : milestone, 18108000,
profiling (15.531 s) : 15531000, 15531000
. : milestone, 15531000,
tracing (14.771 s) : 14771000, 14771000
. : milestone, 14771000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.507 s [15.507 s, 15.507 s]	-
appsec	14.911 s [14.911 s, 14.911 s]	-596.0 ms (-3.8%)
iast	18.67 s [18.67 s, 18.67 s]	3.163 s (20.4%)
iast_GLOBAL	18.245 s [18.245 s, 18.245 s]	2.738 s (17.7%)
profiling	15.131 s [15.131 s, 15.131 s]	-376.0 ms (-2.4%)
tracing	14.869 s [14.869 s, 14.869 s]	-638.0 ms (-4.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.494 s [15.494 s, 15.494 s]	-
appsec	14.821 s [14.821 s, 14.821 s]	-673.0 ms (-4.3%)
iast	18.781 s [18.781 s, 18.781 s]	3.287 s (21.2%)
iast_GLOBAL	18.108 s [18.108 s, 18.108 s]	2.614 s (16.9%)
profiling	15.531 s [15.531 s, 15.531 s]	37.0 ms (0.2%)
tracing	14.771 s [14.771 s, 14.771 s]	-723.0 ms (-4.7%)

datadog-datadog-prod-us1 · 2025-07-31T15:08:33Z

Code coverage: total 57.51%, base diff 0.06%, patch 88.95% (view details)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: e9095b5 | Docs | Was this helpful? Give us feedback!}

PerfectSlayer

Thanks for all the follow up changes!
Nice work, especially around testing 👏

amarziali

It looks good. I left few comments or questions for improvement or clarification

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.52.1` -> `1.53.0` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.53.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.53.0): 1.53.0 ### Components #### Application Security Management (WAF) - 🐛 Fix NullPointerException log in AppSec ([#9355](DataDog/dd-trace-java#9355) - [@jandro996](https://github.com/jandro996)) - ✨ Add Forwarded header as a source to client IP resolution ([#9310](DataDog/dd-trace-java#9310) - [@jandro996](https://github.com/jandro996)) #### Build & Tooling - 📖 Update CODEOWNERS ([#9294](DataDog/dd-trace-java#9294) - [@mhlidd](https://github.com/mhlidd)) - ✨ Add dependency on Maven Central deployment to OCI publish jobs ([#9204](DataDog/dd-trace-java#9204) - [@sarahchen6](https://github.com/sarahchen6)) #### Continuous Integration Visibility - ✨ Upload code coverage reports to Datadog ([#9425](DataDog/dd-trace-java#9425) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Do not follow symlinks by default when building repository index ([#9318](DataDog/dd-trace-java#9318) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Improve tag propagation between test event levels ([#9278](DataDog/dd-trace-java#9278) - [@daniel-mohedano](https://github.com/daniel-mohedano)) #### Data Streams Monitoring - ✨ Create container hash tags propagation ([#9282](DataDog/dd-trace-java#9282) - [@ygree](https://github.com/ygree)) #### Database Monitoring - 🐛 Prevent crash in SQL Server's JDBC when tracing execute methods with generated keys ([#9321](DataDog/dd-trace-java#9321) - [@na-ji](https://github.com/na-ji)) - ✨ Create container hash tags propagation ([#9282](DataDog/dd-trace-java#9282) - [@ygree](https://github.com/ygree)) #### GraalVM native-image - 🐛 Update GraalVM config to reflect TempLocationManager's new package ([#9337](DataDog/dd-trace-java#9337) - [@luneo7](https://github.com/luneo7) - thanks for the contribution!) #### Metrics - 🐛 Add reflect config for CSS and graalvm ([#9449](DataDog/dd-trace-java#9449) - [@amarziali](https://github.com/amarziali)) - ✨ Align CSS health metrics ([#9444](DataDog/dd-trace-java#9444) - [@amarziali](https://github.com/amarziali)) - ✨ Aggregate peer tags for consumer span kind ([#9442](DataDog/dd-trace-java#9442) - [@amarziali](https://github.com/amarziali)) - ✨ Implement health metrics for client stats ([#9377](DataDog/dd-trace-java#9377) - [@amarziali](https://github.com/amarziali)) - ✨ Add peer tags, span kind and trace root flag to MetricKey bucket ([#9178](DataDog/dd-trace-java#9178) - [@bric3](https://github.com/bric3)) #### Profiling - ✨ Create a profile flare reporter (requires tracing or CI vis to be enabled) ([#9433](DataDog/dd-trace-java#9433) - [@MattAlp](https://github.com/MattAlp)) - 🐛 Properly fall-back on UDS profiling URL ([#9401](DataDog/dd-trace-java#9401) - [@jbachorik](https://github.com/jbachorik)) - 🐛 Properly handle trace agent IPv6 URL in profiling ([#9334](DataDog/dd-trace-java#9334) - [@jbachorik](https://github.com/jbachorik)) #### Realtime User Monitoring - 🐛 Avoid multiple injections on dispatch on jboss ([#9392](DataDog/dd-trace-java#9392) - [@amarziali](https://github.com/amarziali)) - ✨ Support async servlet for RUM injection ([#9333](DataDog/dd-trace-java#9333) - [@amarziali](https://github.com/amarziali)) - 🐛 Improve RUM injection matching and avoid truncating responses ([#9323](DataDog/dd-trace-java#9323) - [@amarziali](https://github.com/amarziali)) - 💡 Add telemetry for the RUM injector ([#9267](DataDog/dd-trace-java#9267) - [@sarahchen6](https://github.com/sarahchen6)) - ✨ Make rum injector stream/writer more resilient to errors ([#9184](DataDog/dd-trace-java#9184) - [@amarziali](https://github.com/amarziali)) #### Telemetry - 🐛 Remediate error logs ([#9459](DataDog/dd-trace-java#9459) - [@ygree](https://github.com/ygree)) - ✨ Report config\_id for Hands Off Config files ([#9299](DataDog/dd-trace-java#9299) - [@mtoffl01](https://github.com/mtoffl01)) - ✨ Adding Config Inversion Telemetry component ([#9244](DataDog/dd-trace-java#9244) - [@mhlidd](https://github.com/mhlidd)) #### Trace context propagation - 🧹 Migrate instrumentations to Context API ([#9384](DataDog/dd-trace-java#9384), [#9378](DataDog/dd-trace-java#9378), [#9358](DataDog/dd-trace-java#9358) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🔍 Add baggage propagation telemetry ([#9289](DataDog/dd-trace-java#9289) - [@rachelyangdog](https://github.com/rachelyangdog)) - 🧹 Improve Instrumenter API to use Context instead of Span ([#9211](DataDog/dd-trace-java#9211) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Tracer core - 💡:test\_tube: Enable process level tags for java 21 applications ([#9458](DataDog/dd-trace-java#9458) - [@amarziali](https://github.com/amarziali)) - ✨ Move base and peer service adder before span is published ([#9408](DataDog/dd-trace-java#9408) - [@amarziali](https://github.com/amarziali)) - ✨ Add ErrorPriorities level for manual instrumentation ([#9387](DataDog/dd-trace-java#9387) - [@cecile75](https://github.com/cecile75)) - ✨ Throw InvalidBooleanValueException in ConfigConverter.booleanValueOf ([#9370](DataDog/dd-trace-java#9370) - [@mtoffl01](https://github.com/mtoffl01)) - 🐛 Avoid NPE on featureDiscovery creation ([#9353](DataDog/dd-trace-java#9353) - [@amarziali](https://github.com/amarziali)) - ✨ Stable Config improvements ([#9259](DataDog/dd-trace-java#9259) - [@mtoffl01](https://github.com/mtoffl01)) ### Instrumentations #### gRPC instrumentation - 🐛 Changing GRPC instrumentation codes from `status.code` to `grpc.status.code` ([#9367](DataDog/dd-trace-java#9367) - [@mhlidd](https://github.com/mhlidd)) #### JDBC instrumentation - 🐛 Prevent crash in SQL Server's JDBC when tracing execute methods with generated keys ([#9321](DataDog/dd-trace-java#9321) - [@na-ji](https://github.com/na-ji)) #### Kotlin instrumentation - 🐛 Take defensive copy of original scope stack when restoring on different thread ([#9403](DataDog/dd-trace-java#9403) - [@mcculls](https://github.com/mcculls)) #### OpenTelemetry instrumentation - 🐛 Fix using SpanAttribute annotation on multiple parameters ([#9412](DataDog/dd-trace-java#9412) - [@cecile75](https://github.com/cecile75)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: e23ec866cef91afc6b5226d5f0dc7da538d947e8

sarahchen6 added type: feature request comp: metrics Metrics labels Jul 28, 2025

sarahchen6 force-pushed the sarahchen6/implement-rum-injector-telemetry branch 3 times, most recently from 1e2a4ed to 0687a05Compare July 31, 2025 14:45

sarahchen6 force-pushed the sarahchen6/implement-rum-injector-telemetry branch from dbf1bee to 87c7127Compare August 2, 2025 03:25

PerfectSlayer added comp: rum Realtime User Monitoring and removed comp: metrics Metrics labels Aug 8, 2025

sarahchen6 added 16 commits August 8, 2025 10:47

Add RumInjectorHealthMetrics
d19d27e

Add telemetry collector and methods to RumInjector
e01d1a9

Initialize health metrics and telemetry collector
8c71ef6

Get injectionsucceed count
6c01e10

Add comments
ceb3e11

Reorganize classes
92f0c54

Connect rum injector, telemetry collector, and statsdclient
c5c860f

Add tests
3d4ac53

Get and test metrics for injection failures and skips
3fd498d

Add Content Security Policy and HTTP response size telemetry
14c338b

Add injection duration telemetry
c5b5389

Fix some things
feb40be

Fix content-length retrieval and add test for injection timing
23e0455

Add injection initialization success telemetry
07db174

Fix CoreTracer compilation with InstrumenterConfig
a5352ff

Add tags to all metrics
2a1f676

sarahchen6 force-pushed the sarahchen6/implement-rum-injector-telemetry branch from 7bd7a48 to 2a1f676Compare August 8, 2025 19:21

sarahchen6 added 3 commits August 8, 2025 16:42

Update InjectingPipeOutputStreamTest
b32874c

Tweaks
7a5cd68

Address jacoco coverage and injectingpipeoutstream interface updates
412a3f8

Add lots of improvements from review comments
b1ab871

PerfectSlayer approved these changes Aug 13, 2025
View reviewed changes

sarahchen6 changed the title ~~[WIP] Add telemetry for the RUM injector~~Add telemetry for the RUM injectorAug 13, 2025

sarahchen6 marked this pull request as ready for review August 13, 2025 16:27

sarahchen6 requested review from a team as code owners August 13, 2025 16:27

sarahchen6 requested a review from amarziali August 13, 2025 16:27

sarahchen6 added 2 commits August 13, 2025 12:55

Fix constructors and address review comment
93bd0a4

Merge branch 'master' into sarahchen6/implement-rum-injector-telemetry
5a24d7d

AlexeyKuznetsov-DD reviewed Aug 13, 2025
View reviewed changes

Comment thread.../src/main/java/datadog/trace/bootstrap/instrumentation/buffer/InjectingPipeOutputStream.java Outdated

Clarify bytes written and address review comment
b20d6c4

AlexeyKuznetsov-DD approved these changes Aug 13, 2025
View reviewed changes

amarziali reviewed Aug 18, 2025
View reviewed changes

Comment thread...st-3/src/main/java/datadog/trace/instrumentation/servlet3/RumHttpServletResponseWrapper.java Outdated

amarziali reviewed Aug 18, 2025
View reviewed changes

Comment thread...st-3/src/main/java/datadog/trace/instrumentation/servlet3/RumHttpServletResponseWrapper.java Outdated

amarziali approved these changes Aug 18, 2025
View reviewed changes

sarahchen6 added 3 commits August 18, 2025 16:33

Use dynamic servlet version retrieval
f7607ba

Change injection timing logic
e6afc52

Clean up
6f2dc97

amarziali reviewed Aug 19, 2025
View reviewed changes

Comment threadinternal-api/src/main/java/datadog/trace/api/rum/RumInjectorMetrics.java Outdated

sarahchen6 added 4 commits August 19, 2025 10:39

Use dynamic servlet version retrieval for tagging as well
32372c1

Address merge conflicts
821dc87

Add a telemetry check to HttpServerTest
150ab05

Clean up
e9095b5

amarziali reviewed Aug 20, 2025
View reviewed changes

Comment threadinternal-api/src/main/java/datadog/trace/api/rum/RumInjectorMetrics.java

sarahchen6 merged commit 9bade09 into masterAug 20, 2025
666 of 667 checks passed

sarahchen6 deleted the sarahchen6/implement-rum-injector-telemetry branch August 20, 2025 12:41

github-actionsBot added this to the 1.53.0 milestone Aug 20, 2025

sarahchen6 mentioned this pull request Sep 13, 2025
Refactor RUM telemetry #9526
Merged

Navigation Menu

Uh oh!

Add telemetry for the RUM injector - #9267

Add telemetry for the RUM injector#9267
sarahchen6 merged 33 commits into
masterfrom
sarahchen6/implement-rum-injector-telemetry

sarahchen6 commented Jul 28, 2025•
edited
Loading

Uh oh!

pr-commenterBot commented Jul 28, 2025•
edited
Loading

Uh oh!

datadog-datadog-prod-us1Bot commented Jul 31, 2025•
edited by datadog-official Bot
Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amarziali left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

sarahchen6 commented Jul 28, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenterBot commented Jul 28, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

datadog-datadog-prod-us1Bot commented Jul 31, 2025• edited by datadog-official BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amarziali left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sarahchen6 commented Jul 28, 2025•
edited
Loading

pr-commenterBot commented Jul 28, 2025•
edited
Loading

datadog-datadog-prod-us1Bot commented Jul 31, 2025•
edited by datadog-official Bot
Loading