Gary/llmobs java sdk integration by gary-huang · Pull Request #8519 · DataDog/dd-trace-java

gary-huang · 2025-03-07T00:08:01Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-03-07T00:08:11Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

pr-commenter · 2025-03-07T00:51:42Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/llmobs-java-sdk-integration
git_commit_date	1749573424	1749600946
git_commit_sha	`3853c65`	`8659686`
release_version	1.50.0-SNAPSHOT~3853c65296	1.50.0-SNAPSHOT~8659686578

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1749603521	1749603521
ci_job_id	976126894	976126894
ci_pipeline_id	67408882	67408882
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-63wmvl1-project-304-concurrent-0-zdq0a5r3 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-63wmvl1-project-304-concurrent-0-zdq0a5r3 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 57 metrics, 12 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:petclinic:iast:Debugger	better [-369.937µs; -125.980µs] or [-6.063%; -2.065%]	5.854ms	6.102ms
scenario:startup:petclinic:tracing:Remote Config	worse [+18.196µs; +72.418µs] or [+2.595%; +10.326%]	746.638µs	701.331µs

Startup time reports for petclinic

gantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.023 s) : 0, 1023271
Total [baseline] (11.027 s) : 0, 11026938
Agent [candidate] (1.032 s) : 0, 1031768
Total [candidate] (11.178 s) : 0, 11177784
section appsec
Agent [baseline] (1.161 s) : 0, 1161224
Total [baseline] (11.206 s) : 0, 11205645
Agent [candidate] (1.163 s) : 0, 1163397
Total [candidate] (11.225 s) : 0, 11224752
section iast
Agent [baseline] (1.166 s) : 0, 1166290
Total [baseline] (11.38 s) : 0, 11380416
Agent [candidate] (1.15 s) : 0, 1150314
Total [candidate] (11.317 s) : 0, 11317378
section profiling
Agent [baseline] (1.262 s) : 0, 1262208
Total [baseline] (11.454 s) : 0, 11454127
Agent [candidate] (1.272 s) : 0, 1272384
Total [candidate] (11.419 s) : 0, 11419143

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.023 s	-
Agent	appsec	1.161 s	137.953 ms (13.5%)
Agent	iast	1.166 s	143.019 ms (14.0%)
Agent	profiling	1.262 s	238.938 ms (23.4%)
Total	tracing	11.027 s	-
Total	appsec	11.206 s	178.707 ms (1.6%)
Total	iast	11.38 s	353.478 ms (3.2%)
Total	profiling	11.454 s	427.189 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.032 s	-
Agent	appsec	1.163 s	131.629 ms (12.8%)
Agent	iast	1.15 s	118.545 ms (11.5%)
Agent	profiling	1.272 s	240.616 ms (23.3%)
Total	tracing	11.178 s	-
Total	appsec	11.225 s	46.968 ms (0.4%)
Total	iast	11.317 s	139.594 ms (1.2%)
Total	profiling	11.419 s	241.359 ms (2.2%)

gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.017 ms) : 0, 683017
BytebuddyAgent [candidate] (689.391 ms) : 0, 689391
GlobalTracer [baseline] (240.3 ms) : 0, 240300
GlobalTracer [candidate] (242.316 ms) : 0, 242316
AppSec [baseline] (58.322 ms) : 0, 58322
AppSec [candidate] (57.303 ms) : 0, 57303
Debugger [baseline] (6.097 ms) : 0, 6097
Debugger [candidate] (6.243 ms) : 0, 6243
Remote Config [baseline] (701.331 µs) : 0, 701
Remote Config [candidate] (746.638 µs) : 0, 747
Telemetry [baseline] (11.225 ms) : 0, 11225
Telemetry [candidate] (12.038 ms) : 0, 12038
section appsec
BytebuddyAgent [baseline] (698.887 ms) : 0, 698887
BytebuddyAgent [candidate] (701.44 ms) : 0, 701440
GlobalTracer [baseline] (237.494 ms) : 0, 237494
GlobalTracer [candidate] (237.466 ms) : 0, 237466
IAST [baseline] (21.829 ms) : 0, 21829
IAST [candidate] (21.901 ms) : 0, 21901
AppSec [baseline] (176.478 ms) : 0, 176478
AppSec [candidate] (175.956 ms) : 0, 175956
Debugger [baseline] (5.998 ms) : 0, 5998
Debugger [candidate] (5.974 ms) : 0, 5974
Remote Config [baseline] (625.524 µs) : 0, 626
Remote Config [candidate] (628.908 µs) : 0, 629
Telemetry [baseline] (7.346 ms) : 0, 7346
Telemetry [candidate] (7.364 ms) : 0, 7364
section iast
BytebuddyAgent [baseline] (813.131 ms) : 0, 813131
BytebuddyAgent [candidate] (802.305 ms) : 0, 802305
GlobalTracer [baseline] (233.379 ms) : 0, 233379
GlobalTracer [candidate] (230.729 ms) : 0, 230729
IAST [baseline] (26.064 ms) : 0, 26064
IAST [candidate] (26.879 ms) : 0, 26879
AppSec [baseline] (55.187 ms) : 0, 55187
AppSec [candidate] (52.51 ms) : 0, 52510
Debugger [baseline] (6.102 ms) : 0, 6102
Debugger [candidate] (5.854 ms) : 0, 5854
Remote Config [baseline] (617.528 µs) : 0, 618
Remote Config [candidate] (585.043 µs) : 0, 585
Telemetry [baseline] (8.087 ms) : 0, 8087
Telemetry [candidate] (7.853 ms) : 0, 7853
section profiling
BytebuddyAgent [baseline] (674.602 ms) : 0, 674602
BytebuddyAgent [candidate] (680.088 ms) : 0, 680088
GlobalTracer [baseline] (359.235 ms) : 0, 359235
GlobalTracer [candidate] (360.791 ms) : 0, 360791
AppSec [baseline] (61.107 ms) : 0, 61107
AppSec [candidate] (61.749 ms) : 0, 61749
Debugger [baseline] (5.937 ms) : 0, 5937
Debugger [candidate] (6.087 ms) : 0, 6087
Remote Config [baseline] (654.739 µs) : 0, 655
Remote Config [candidate] (642.767 µs) : 0, 643
Telemetry [baseline] (8.022 ms) : 0, 8022
Telemetry [candidate] (8.216 ms) : 0, 8216
ProfilingAgent [baseline] (102.015 ms) : 0, 102015
ProfilingAgent [candidate] (103.722 ms) : 0, 103722
Profiling [baseline] (102.039 ms) : 0, 102039
Profiling [candidate] (103.748 ms) : 0, 103748

Startup time reports for insecure-bank

gantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.023 s) : 0, 1022582
Total [baseline] (8.531 s) : 0, 8531266
Agent [candidate] (1.024 s) : 0, 1023961
Total [candidate] (8.5 s) : 0, 8500152
section iast
Agent [baseline] (1.147 s) : 0, 1146996
Total [baseline] (9.172 s) : 0, 9171605
Agent [candidate] (1.164 s) : 0, 1163873
Total [candidate] (9.168 s) : 0, 9168004
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.15 s) : 0, 1150284
Total [baseline] (9.127 s) : 0, 9126838
Agent [candidate] (1.157 s) : 0, 1157251
Total [candidate] (9.146 s) : 0, 9146232
section iast_TELEMETRY_OFF
Agent [baseline] (1.144 s) : 0, 1144450
Total [baseline] (9.208 s) : 0, 9208130
Agent [candidate] (1.16 s) : 0, 1160437
Total [candidate] (9.26 s) : 0, 9259806

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.023 s	-
Agent	iast	1.147 s	124.414 ms (12.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.15 s	127.702 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.144 s	121.868 ms (11.9%)
Total	tracing	8.531 s	-
Total	iast	9.172 s	640.338 ms (7.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.127 s	595.572 ms (7.0%)
Total	iast_TELEMETRY_OFF	9.208 s	676.864 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.024 s	-
Agent	iast	1.164 s	139.912 ms (13.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.157 s	133.29 ms (13.0%)
Agent	iast_TELEMETRY_OFF	1.16 s	136.476 ms (13.3%)
Total	tracing	8.5 s	-
Total	iast	9.168 s	667.851 ms (7.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.146 s	646.08 ms (7.6%)
Total	iast_TELEMETRY_OFF	9.26 s	759.654 ms (8.9%)

gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.944 ms) : 0, 681944
BytebuddyAgent [candidate] (683.705 ms) : 0, 683705
GlobalTracer [baseline] (239.31 ms) : 0, 239310
GlobalTracer [candidate] (240.044 ms) : 0, 240044
AppSec [baseline] (54.795 ms) : 0, 54795
AppSec [candidate] (57.013 ms) : 0, 57013
Debugger [baseline] (6.199 ms) : 0, 6199
Debugger [candidate] (6.172 ms) : 0, 6172
Remote Config [baseline] (747.433 µs) : 0, 747
Remote Config [candidate] (736.136 µs) : 0, 736
Telemetry [baseline] (15.997 ms) : 0, 15997
Telemetry [candidate] (12.769 ms) : 0, 12769
section iast
BytebuddyAgent [baseline] (800.197 ms) : 0, 800197
BytebuddyAgent [candidate] (812.522 ms) : 0, 812522
GlobalTracer [baseline] (229.923 ms) : 0, 229923
GlobalTracer [candidate] (232.718 ms) : 0, 232718
IAST [baseline] (27.072 ms) : 0, 27072
IAST [candidate] (26.639 ms) : 0, 26639
AppSec [baseline] (52.013 ms) : 0, 52013
AppSec [candidate] (53.724 ms) : 0, 53724
Debugger [baseline] (5.92 ms) : 0, 5920
Debugger [candidate] (5.992 ms) : 0, 5992
Remote Config [baseline] (592.446 µs) : 0, 592
Remote Config [candidate] (607.013 µs) : 0, 607
Telemetry [baseline] (7.856 ms) : 0, 7856
Telemetry [candidate] (7.983 ms) : 0, 7983
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.916 ms) : 0, 801916
BytebuddyAgent [candidate] (807.601 ms) : 0, 807601
GlobalTracer [baseline] (230.654 ms) : 0, 230654
GlobalTracer [candidate] (231.479 ms) : 0, 231479
IAST [baseline] (27.049 ms) : 0, 27049
IAST [candidate] (27.825 ms) : 0, 27825
AppSec [baseline] (52.696 ms) : 0, 52696
AppSec [candidate] (51.969 ms) : 0, 51969
Debugger [baseline] (5.945 ms) : 0, 5945
Debugger [candidate] (6.033 ms) : 0, 6033
Remote Config [baseline] (599.477 µs) : 0, 599
Remote Config [candidate] (609.286 µs) : 0, 609
Telemetry [baseline] (7.924 ms) : 0, 7924
Telemetry [candidate] (8.062 ms) : 0, 8062
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (796.921 ms) : 0, 796921
BytebuddyAgent [candidate] (809.101 ms) : 0, 809101
GlobalTracer [baseline] (230.542 ms) : 0, 230542
GlobalTracer [candidate] (233.031 ms) : 0, 233031
IAST [baseline] (25.683 ms) : 0, 25683
IAST [candidate] (30.808 ms) : 0, 30808
AppSec [baseline] (53.421 ms) : 0, 53421
AppSec [candidate] (49.13 ms) : 0, 49130
Debugger [baseline] (5.971 ms) : 0, 5971
Debugger [candidate] (6.024 ms) : 0, 6024
Remote Config [baseline] (600.401 µs) : 0, 600
Remote Config [candidate] (596.605 µs) : 0, 597
Telemetry [baseline] (7.79 ms) : 0, 7790
Telemetry [candidate] (8.03 ms) : 0, 8030

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-06-11T00:37:54	2025-06-11T00:42:53
git_branch	master	gary/llmobs-java-sdk-integration
git_commit_date	1749573424	1749600946
git_commit_sha	`3853c65`	`8659686`
release_version	1.50.0-SNAPSHOT~3853c65296	1.50.0-SNAPSHOT~8659686578
start_time	2025-06-11T00:37:23	2025-06-11T00:42:04

See matching parameters

	Baseline	Candidate
application	petclinic	petclinic
ci_job_date	1749602573	1749602573
ci_job_id	976126895	976126895
ci_pipeline_id	67408882	67408882
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-63wmvl1-project-304-concurrent-1-1qo0btoq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-63wmvl1-project-304-concurrent-1-1qo0btoq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
thresholds_or_results	results	results
variant	appsec	appsec

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 0 metrics, 11 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:code_origins	unstable [+55.246ms; +60.060ms] or [+628.087%; +682.818%]	worse [-512.864op/s; -476.655op/s] or [-91.209%; -84.769%]	66.449ms	67.538op/s	8.796ms	562.298op/s
scenario:load:petclinic:no_agent	unstable [+21.958ms; +22.781ms] or [+287.620%; +298.401%]	worse [-511.258op/s; -452.517op/s] or [-79.245%; -70.140%]	30.004ms	163.274op/s	7.635ms	645.161op/s
scenario:load:petclinic:tracing	unstable [+28.601ms; +29.586ms] or [+345.956%; +357.865%]	worse [-482.714op/s; -447.876op/s] or [-80.855%; -75.019%]	37.361ms	131.720op/s	8.267ms	597.015op/s

Request duration reports for petclinic

gantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section baseline
no_agent (7.635 ms) : 7612, 7657
. : milestone, 7635,
appsec (1.708 ms) : 1638, 1778
. : milestone, 1708,
appsec_no_iast (1.004 ms) : 944, 1064
. : milestone, 1004,
code_origins (8.796 ms) : 8765, 8827
. : milestone, 8796,
iast (292.044 µs) : 256, 328
. : milestone, 292,
profiling (10.235 ms) : 10177, 10293
. : milestone, 10235,
tracing (8.267 ms) : 8241, 8293
. : milestone, 8267,
section candidate
no_agent (30.004 ms) : 29464, 30545
. : milestone, 30004,
appsec (51.757 ms) : 50679, 52836
. : milestone, 51757,
appsec_no_iast (56.298 ms) : 54934, 57662
. : milestone, 56298,
code_origins (66.449 ms) : 63286, 69612
. : milestone, 66449,
iast (62.395 ms) : 60506, 64284
. : milestone, 62395,
profiling (44.966 ms) : 44121, 45810
. : milestone, 44966,
tracing (37.361 ms) : 36714, 38007
. : milestone, 37361,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	7.635 ms [7.612 ms, 7.657 ms]	-
appsec	1.708 ms [1.638 ms, 1.778 ms]	-5.926 ms (-77.6%)
appsec_no_iast	1.004 ms [943.661 µs, 1.064 ms]	-6.631 ms (-86.9%)
code_origins	8.796 ms [8.765 ms, 8.827 ms]	1.161 ms (15.2%)
iast	292.044 µs [255.903 µs, 328.185 µs]	-7.342 ms (-96.2%)
profiling	10.235 ms [10.177 ms, 10.293 ms]	2.601 ms (34.1%)
tracing	8.267 ms [8.241 ms, 8.293 ms]	632.72 µs (8.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	30.004 ms [29.464 ms, 30.545 ms]	-
appsec	51.757 ms [50.679 ms, 52.836 ms]	21.753 ms (72.5%)
appsec_no_iast	56.298 ms [54.934 ms, 57.662 ms]	26.294 ms (87.6%)
code_origins	66.449 ms [63.286 ms, 69.612 ms]	36.445 ms (121.5%)
iast	62.395 ms [60.506 ms, 64.284 ms]	32.391 ms (108.0%)
profiling	44.966 ms [44.121 ms, 45.81 ms]	14.961 ms (49.9%)
tracing	37.361 ms [36.714 ms, 38.007 ms]	7.356 ms (24.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/llmobs-java-sdk-integration
git_commit_date	1749573424	1749600946
git_commit_sha	`3853c65`	`8659686`
release_version	1.50.0-SNAPSHOT~3853c65296	1.50.0-SNAPSHOT~8659686578

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1749603119	1749603119
ci_job_id	976126896	976126896
ci_pipeline_id	67408882	67408882
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-pz8nwqvr-project-304-concurrent-0-zas4l10v 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-pz8nwqvr-project-304-concurrent-0-zas4l10v 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section baseline
no_agent (15.402 s) : 15402000, 15402000
. : milestone, 15402000,
appsec (14.838 s) : 14838000, 14838000
. : milestone, 14838000,
iast (18.852 s) : 18852000, 18852000
. : milestone, 18852000,
iast_GLOBAL (18.048 s) : 18048000, 18048000
. : milestone, 18048000,
profiling (15.803 s) : 15803000, 15803000
. : milestone, 15803000,
tracing (14.975 s) : 14975000, 14975000
. : milestone, 14975000,
section candidate
no_agent (15.203 s) : 15203000, 15203000
. : milestone, 15203000,
appsec (14.745 s) : 14745000, 14745000
. : milestone, 14745000,
iast (19.209 s) : 19209000, 19209000
. : milestone, 19209000,
iast_GLOBAL (18.02 s) : 18020000, 18020000
. : milestone, 18020000,
profiling (15.343 s) : 15343000, 15343000
. : milestone, 15343000,
tracing (14.814 s) : 14814000, 14814000
. : milestone, 14814000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.402 s [15.402 s, 15.402 s]	-
appsec	14.838 s [14.838 s, 14.838 s]	-564.0 ms (-3.7%)
iast	18.852 s [18.852 s, 18.852 s]	3.45 s (22.4%)
iast_GLOBAL	18.048 s [18.048 s, 18.048 s]	2.646 s (17.2%)
profiling	15.803 s [15.803 s, 15.803 s]	401.0 ms (2.6%)
tracing	14.975 s [14.975 s, 14.975 s]	-427.0 ms (-2.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.203 s [15.203 s, 15.203 s]	-
appsec	14.745 s [14.745 s, 14.745 s]	-458.0 ms (-3.0%)
iast	19.209 s [19.209 s, 19.209 s]	4.006 s (26.4%)
iast_GLOBAL	18.02 s [18.02 s, 18.02 s]	2.817 s (18.5%)
profiling	15.343 s [15.343 s, 15.343 s]	140.0 ms (0.9%)
tracing	14.814 s [14.814 s, 14.814 s]	-389.0 ms (-2.6%)

Execution time for tomcat

gantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~3853c65296
dateFormat X
axisFormat %s
section baseline
no_agent (1.485 ms) : 1473, 1496
. : milestone, 1485,
appsec (2.418 ms) : 2369, 2467
. : milestone, 2418,
iast (2.199 ms) : 2138, 2260
. : milestone, 2199,
iast_GLOBAL (2.24 ms) : 2178, 2301
. : milestone, 2240,
profiling (2.039 ms) : 1990, 2089
. : milestone, 2039,
tracing (2.021 ms) : 1973, 2068
. : milestone, 2021,
section candidate
no_agent (1.481 ms) : 1469, 1493
. : milestone, 1481,
appsec (2.423 ms) : 2375, 2472
. : milestone, 2423,
iast (2.198 ms) : 2137, 2260
. : milestone, 2198,
iast_GLOBAL (2.238 ms) : 2177, 2300
. : milestone, 2238,
profiling (2.505 ms) : 2331, 2679
. : milestone, 2505,
tracing (2.023 ms) : 1976, 2071
. : milestone, 2023,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.485 ms [1.473 ms, 1.496 ms]	-
appsec	2.418 ms [2.369 ms, 2.467 ms]	933.292 µs (62.9%)
iast	2.199 ms [2.138 ms, 2.26 ms]	714.428 µs (48.1%)
iast_GLOBAL	2.24 ms [2.178 ms, 2.301 ms]	755.167 µs (50.9%)
profiling	2.039 ms [1.99 ms, 2.089 ms]	554.963 µs (37.4%)
tracing	2.021 ms [1.973 ms, 2.068 ms]	536.191 µs (36.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.493 ms]	-
appsec	2.423 ms [2.375 ms, 2.472 ms]	942.396 µs (63.6%)
iast	2.198 ms [2.137 ms, 2.26 ms]	717.308 µs (48.4%)
iast_GLOBAL	2.238 ms [2.177 ms, 2.3 ms]	757.407 µs (51.1%)
profiling	2.505 ms [2.331 ms, 2.679 ms]	1.024 ms (69.1%)
tracing	2.023 ms [1.976 ms, 2.071 ms]	542.358 µs (36.6%)

github-actions · 2025-03-07T03:37:40Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T04:49:20Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T12:33:38Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T12:36:22Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-03-07T14:09:24Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-04-21T14:01:01Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

github-actions · 2025-04-21T14:19:05Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

…g provided in the override

* add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners

github-actions · 2025-05-14T16:15:07Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

* add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners

* add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * spotless * add APIs for llm obs sdk (#8135) * add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners * rename ID to Id according to java naming conventions * Undo change to integrations-core submodule * fix build gradle * rm empty line * fix test

github-actions · 2025-06-10T23:30:20Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

…gration

github-actions · 2025-10-01T03:19:23Z

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

github-actions · 2025-11-01T03:19:37Z

This pull request has been closed because it has not had activity over the past quarter. Feel free to reopen the PR if you are still working on it.

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 058de60 to 612fd71Compare March 7, 2025 03:37

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from e320771 to b14ebedCompare March 7, 2025 12:36

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from b14ebed to 48b1a07Compare March 7, 2025 14:09

gary-huang added 14 commits April 10, 2025 15:09

add APIs for llm obs
5134311

add llm message class to support llm spans
b5c7a0a

add llm message class to support llm spans
7f8f586

impl llmobs agent and llmobs apis
9206dbe

support llm messages with tool calls
f5efd26

handle default model name and provider
76740e9

rm unneeded file
5e66a2f

impl llmobs agent and llmobs apis
75290af

impl llmobs agent
7b2db7c

working writer
d76c94d

add support for llm message and tool calls
e0b4510

impl llmobs agent and llmobs apis
87cdaf1

use new ctx api to track parent span
f78b2c5

add api for evals
5fc9069

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 48b1a07 to bb6d246Compare April 21, 2025 14:00

working impl supporting both agentless and agent
a74e456

handle null tags and default to default ml app if null or empty strin…
19d95d7
…g provided in the override

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 0735fae to 80671d1Compare April 21, 2025 14:38

gary-huang added 5 commits April 23, 2025 14:56

spotless
de0d5a5

add APIs for llm obs sdk (#8135)
82b8418
* add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners

Merge branch 'gary/llmobs-sdk-merge' into gary/impl-llm-obs-api
f1d9d17

rename ID to Id according to java naming conventions
0a790c3

Merge branch 'gary/impl-llm-obs-api' into gary/add-llm-obs-writer
d68665d

gary-huang added 10 commits May 14, 2025 14:17

Merge branch 'master' into gary/impl-llm-obs-api
0d77da2

Undo change to integrations-core submodule
c1c38e4

add APIs for llm obs sdk (#8135)
2be1b73
* add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners

fix build gradle
d01034a

Merge branch 'gary/llmobs-sdk-merge' into gary/impl-llm-obs-api
7a74cf6

rm empty line
23d6923

fix test
9ac1ab7

wip add test
e382f22

Merge branch 'gary/impl-llm-obs-api' into gary/add-llm-obs-writer
613e6c2

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from a8f7f7e to bfa0c5bCompare June 10, 2025 23:30

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from bfa0c5b to 08ef55aCompare June 10, 2025 23:51

use proper convention like Id instead of ID
4739666

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from 08ef55a to c8ece6fCompare June 11, 2025 00:03

gary-huang added 4 commits June 10, 2025 20:12

Merge branch 'master' into gary/llmobs-sdk-merge
993cdbe

Merge branch 'gary/add-llm-obs-writer' into gary/llmobs-java-sdk-inte…
20d2760
…gration

Merge branch 'gary/use-ctx-api' into gary/llmobs-java-sdk-integration
8a9d17d

Merge branch 'gary/submit-evals-2' into gary/llmobs-java-sdk-integration
8659686

gary-huang force-pushed the gary/llmobs-java-sdk-integration branch from c8ece6f to 8659686Compare June 11, 2025 00:15

github-actionsBot added the tag: stale Stale pull requests label Oct 1, 2025

github-actionsBot closed this Nov 1, 2025

Navigation Menu

Uh oh!

Gary/llmobs java sdk integration - #8519

Gary/llmobs java sdk integration#8519
gary-huang wants to merge 37 commits into
masterfrom
gary/llmobs-java-sdk-integration

gary-huang commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

pr-commenterBot commented Mar 7, 2025•
edited
Loading

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Apr 21, 2025

Uh oh!

github-actionsBot commented Apr 21, 2025

Uh oh!

github-actionsBot commented May 14, 2025

Uh oh!

github-actionsBot commented Jun 10, 2025

Uh oh!

github-actionsBot commented Oct 1, 2025

Uh oh!

github-actionsBot commented Nov 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gary-huang commented Mar 7, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

pr-commenterBot commented Mar 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Mar 7, 2025

Uh oh!

github-actionsBot commented Apr 21, 2025

Uh oh!

github-actionsBot commented Apr 21, 2025

Uh oh!

github-actionsBot commented May 14, 2025

Uh oh!

github-actionsBot commented Jun 10, 2025

Uh oh!

github-actionsBot commented Oct 1, 2025

Uh oh!

github-actionsBot commented Nov 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenterBot commented Mar 7, 2025•
edited
Loading