Skip to content

[Repro] Standalone LLMObs sampling drop on top of #10989 - #11249

Closed
FouadWahabi wants to merge 10 commits into
masterfrom
wahabi/llmobs-standalone-sampling-repro
Closed

[Repro] Standalone LLMObs sampling drop on top of #10989#11249
FouadWahabi wants to merge 10 commits into
masterfrom
wahabi/llmobs-standalone-sampling-repro

Conversation

@FouadWahabi

@FouadWahabiFouadWahabi commented Apr 30, 2026

Copy link
Copy Markdown

What

Adds a failing smoke test on top of #10989 demonstrating that standalone LLMObs spans (no surrounding APM scope) get silently dropped by the new StandaloneSampler.

Why

In DDLLMObsSpan (dd-java-agent/agent-llmobs/.../DDLLMObsSpan.java):

TraceSegmentsegment = AgentTracer.get().getTraceSegment();
if (segment != null) {
segment.setTagTop(Tags.PROPAGATED_TRACE_SOURCE, ProductTraceSource.LLMOBS);
}

getTraceSegment() reads activeSpan() (CoreTracer.java:1505-1515). But LLMObsContext.attach(span.context()) does not activate an AgentScope — it writes to its own ContextKey. So when LLMObs is used without a surrounding APM scope (CLI, batch, worker, fresh thread, before any HTTP handler), activeSpan() is null, segment is null, and the LLMOBS bit is never propagated to the local root. StandaloneSampler only checks the root's traceSource bitfield → trace is dropped.

The existing LlmObsApmDisabledSmokeTest does not catch this because every LLMObs call runs inside a Spring servlet handler, so the request span is the active scope and is the local root for the LLMObs trace.

How this repro works

  • New endpoint /rest-api/llmobs/standalone that runs LLMObs.startLLMSpan(...) on a fresh thread (no inherited APM scope).
  • New test asserts the resulting trace:
    • exists,
    • has the LLMOBS bit (0x20 → \"20\") in _dd.p.ts on the root,
    • has SAMPLER_KEEP.

On top of #10989 the test fails because the trace receives SAMPLER_DROP and _dd.p.ts is missing.

Suggested fix

Use the LLMObs span's own context (already a TraceSegment via DDSpanContext) instead of the active-scope segment:

if (span.context() instanceofTraceSegment) {
((TraceSegment) span.context()).setTagTop(Tags.PROPAGATED_TRACE_SOURCE, ProductTraceSource.LLMOBS);
}

DDSpanContext.setTagTop already routes through getRootSpanContextOrThis() (DDSpanContext.java:1363), so it correctly marks the local root regardless of which scope is active.

Test plan

  • Run ./gradlew :dd-smoke-tests:apm-tracing-disabled:test --tests \"*standalone LLMObs*\" against this branch — expected to fail.
  • Apply the fix above on top — expected to pass.

matsumo-andand others added 10 commits April 29, 2026 00:43
@matsumo-and
Fix NullPointerException in LLMObsSystem when tracing is disabled
dd7a96c
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@matsumo-and
Support DD_APM_TRACING_ENABLED=false with DD_LLMOBS_ENABLED=true
c27b597
Fixes#10051
When DD_APM_TRACING_ENABLED=false is set, APM tracing should be disabled
while allowing other products like LLM Observability to function. Previously,
setting DD_APM_TRACING_ENABLED=false would inadvertently disable LLMObs or
allow APM traces to leak through when no other products were enabled.
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@matsumo-and
Improve LLMObs trace-disabled smoke test
762717e
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@matsumo-and
format: spotless apply
2cdbdd7
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@matsumo-and
Handle both LLMObs and ASM enabled in standalone sampler
3df930e
When APM tracing is disabled but both LLMObs and ASM are enabled,
the previous code returned LlmObsStandaloneSampler which never sent
the 1 APM trace/minute required by ASM for billing and service catalog.
Introduce LlmObsAndAsmStandaloneSampler that keeps all LLMObs and
ASM traces while rate-limiting plain APM traces to 1 per minute.
Also clarify the log message for the ASM-only standalone case.
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@matsumo-and
Unify standalone samplers into a single StandaloneSampler
1a0d979
Replace the three separate sampler classes (AsmStandaloneSampler,
LlmObsStandaloneSampler, LlmObsAndAsmStandaloneSampler) with a single
StandaloneSampler that iterates over a list of active StandaloneProduct
entries, making it trivially extensible to future products.
Also add ProductTraceSource.isAnyStandaloneProductMarked() to simplify
the TraceCollector force-keep bypass check.
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@matsumo-and
Improve standalone sampler test coverage
e9010e7
- Replace individual SamplerTest cases with @unroll matrix that covers all
product-flag combinations and asserts activeProducts contents directly;
add package-private getActiveProducts() getter to StandaloneSampler to
enable this
- Add StandaloneSamplerTest case for spans with both LLMOBS and ASM bits set
simultaneously, verifying LLMOBS wins via list ordering
- Add TraceCollectorTest exercising the full span-finish → CoreTracer write
path to verify that setSamplingPriorityIfNecessary skips the sampler when
APM is disabled, a standalone product flag is set, and priority is already
non-UNSET
Signed-off-by: matsumo-and <yh134.toisanda@gmail.com>
@FouadWahabi
Add smoke test reproducing standalone LLMObs sampling drop
aabdb06
When an LLMObs span is created without an active surrounding APM
AgentScope, DDLLMObsSpan calls AgentTracer.get().getTraceSegment()
which reads activeSpan() — but LLMObsContext.attach does not activate
an AgentScope, so the segment is null and the LLMOBS trace-source bit
is never set on the local root. StandaloneSampler then drops the trace
because it only checks the root's traceSource bitfield.
Add a controller endpoint that creates the LLMObs span on a fresh
thread (no inherited APM scope) and a smoke test that asserts the
standalone LLMObs trace is kept and carries the LLMOBS bit (0x20) in
_dd.p.ts. The test fails on top of the current PR and will pass once
the bit is propagated regardless of active scope.
@FouadWahabi
Implement abstract traceAgentProtocolVersion() in LLMObs smoke tests
0e45599
Required by AbstractApmTracingDisabledSmokeTest. Without this override,
the smoke test sources fail to compile.
@FouadWahabi
Add braces to single-statement if blocks in SamplerTest
f4dbbf9
CodeNarc IfStatementBraces rule requires braces even for single
statements.
@pr-commenter

pr-commenterBot commented Apr 30, 2026

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

BaselineCandidate
baseline_or_candidatebaselinecandidate
git_branchmasterwahabi/llmobs-standalone-sampling-repro
git_commit_date17773868371777574060
git_commit_shae6cac64f4dbbf9
release_version1.62.0-SNAPSHOT~e6cac64dfd1.62.0-SNAPSHOT~f4dbbf9945
See matching parameters
BaselineCandidate
applicationinsecure-bankinsecure-bank
ci_job_date17775757981777575798
ci_job_id16475485361647548536
ci_pipeline_id110780158110780158
cpu_modelIntel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHzIntel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_versionLinux runner-zfyrx7zua-project-304-concurrent-0-wzv6fx18 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/LinuxLinux runner-zfyrx7zua-project-304-concurrent-0-wzv6fx18 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
moduleAgentAgent
parentNoneNone

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 11 unstable metrics.

scenarioΔ mean execution_timecandidate mean execution_timebaseline mean execution_time
scenario:startup:petclinic:profiling:Remote Configbetter
[-42.972µs; -14.052µs] or [-7.123%; -2.329%]		574.793µs603.305µs
Startup time reports for insecure-bank
gantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064377
Total [baseline] (8.841 s) : 0, 8841160
Agent [candidate] (1.065 s) : 0, 1064691
Total [candidate] (8.842 s) : 0, 8842147
section iast
Agent [baseline] (1.244 s) : 0, 1244463
Total [baseline] (9.497 s) : 0, 9496974
Agent [candidate] (1.239 s) : 0, 1239490
Total [candidate] (9.489 s) : 0, 9489254
Loading
  • baseline results
ModuleVariantDurationΔ tracing
Agenttracing1.064 s-
Agentiast1.244 s180.085 ms (16.9%)
Totaltracing8.841 s-
Totaliast9.497 s655.814 ms (7.4%)
  • candidate results
ModuleVariantDurationΔ tracing
Agenttracing1.065 s-
Agentiast1.239 s174.798 ms (16.4%)
Totaltracing8.842 s-
Totaliast9.489 s647.107 ms (7.3%)
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.241 ms) : 0, 1241
crashtracking [candidate] (1.222 ms) : 0, 1222
BytebuddyAgent [baseline] (635.868 ms) : 0, 635868
BytebuddyAgent [candidate] (635.128 ms) : 0, 635128
AgentMeter [baseline] (29.48 ms) : 0, 29480
AgentMeter [candidate] (29.393 ms) : 0, 29393
GlobalTracer [baseline] (249.399 ms) : 0, 249399
GlobalTracer [candidate] (249.563 ms) : 0, 249563
AppSec [baseline] (32.76 ms) : 0, 32760
AppSec [candidate] (33.007 ms) : 0, 33007
Debugger [baseline] (59.959 ms) : 0, 59959
Debugger [candidate] (60.244 ms) : 0, 60244
Remote Config [baseline] (602.938 µs) : 0, 603
Remote Config [candidate] (618.222 µs) : 0, 618
Telemetry [baseline] (8.389 ms) : 0, 8389
Telemetry [candidate] (9.87 ms) : 0, 9870
Flare Poller [baseline] (10.636 ms) : 0, 10636
Flare Poller [candidate] (9.723 ms) : 0, 9723
section iast
crashtracking [baseline] (1.232 ms) : 0, 1232
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (823.708 ms) : 0, 823708
BytebuddyAgent [candidate] (820.62 ms) : 0, 820620
AgentMeter [baseline] (11.284 ms) : 0, 11284
AgentMeter [candidate] (11.276 ms) : 0, 11276
GlobalTracer [baseline] (237.865 ms) : 0, 237865
GlobalTracer [candidate] (237.198 ms) : 0, 237198
AppSec [baseline] (31.376 ms) : 0, 31376
AppSec [candidate] (30.428 ms) : 0, 30428
Debugger [baseline] (62.467 ms) : 0, 62467
Debugger [candidate] (61.989 ms) : 0, 61989
Remote Config [baseline] (536.068 µs) : 0, 536
Remote Config [candidate] (519.007 µs) : 0, 519
Telemetry [baseline] (7.97 ms) : 0, 7970
Telemetry [candidate] (7.867 ms) : 0, 7867
Flare Poller [baseline] (3.351 ms) : 0, 3351
Flare Poller [candidate] (3.343 ms) : 0, 3343
IAST [baseline] (28.612 ms) : 0, 28612
IAST [candidate] (29.05 ms) : 0, 29050
Loading
Startup time reports for petclinic
gantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062541
Total [baseline] (11.002 s) : 0, 11001881
Agent [candidate] (1.07 s) : 0, 1070103
Total [candidate] (11.029 s) : 0, 11029254
section appsec
Agent [baseline] (1.266 s) : 0, 1265796
Total [baseline] (11.022 s) : 0, 11021523
Agent [candidate] (1.266 s) : 0, 1265670
Total [candidate] (11.072 s) : 0, 11071694
section iast
Agent [baseline] (1.247 s) : 0, 1247458
Total [baseline] (11.319 s) : 0, 11319043
Agent [candidate] (1.254 s) : 0, 1253852
Total [candidate] (11.313 s) : 0, 11313446
section profiling
Agent [baseline] (1.198 s) : 0, 1198065
Total [baseline] (10.958 s) : 0, 10957704
Agent [candidate] (1.192 s) : 0, 1192250
Total [candidate] (11.033 s) : 0, 11033130
Loading
  • baseline results
ModuleVariantDurationΔ tracing
Agenttracing1.063 s-
Agentappsec1.266 s203.255 ms (19.1%)
Agentiast1.247 s184.917 ms (17.4%)
Agentprofiling1.198 s135.524 ms (12.8%)
Totaltracing11.002 s-
Totalappsec11.022 s19.642 ms (0.2%)
Totaliast11.319 s317.162 ms (2.9%)
Totalprofiling10.958 s-44.177 ms (-0.4%)
  • candidate results
ModuleVariantDurationΔ tracing
Agenttracing1.07 s-
Agentappsec1.266 s195.567 ms (18.3%)
Agentiast1.254 s183.749 ms (17.2%)
Agentprofiling1.192 s122.147 ms (11.4%)
Totaltracing11.029 s-
Totalappsec11.072 s42.44 ms (0.4%)
Totaliast11.313 s284.192 ms (2.6%)
Totalprofiling11.033 s3.876 ms (0.0%)
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.221 ms) : 0, 1221
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (634.362 ms) : 0, 634362
BytebuddyAgent [candidate] (639.534 ms) : 0, 639534
AgentMeter [baseline] (29.378 ms) : 0, 29378
AgentMeter [candidate] (29.651 ms) : 0, 29651
GlobalTracer [baseline] (248.631 ms) : 0, 248631
GlobalTracer [candidate] (250.516 ms) : 0, 250516
AppSec [baseline] (32.662 ms) : 0, 32662
AppSec [candidate] (33.071 ms) : 0, 33071
Debugger [baseline] (60.595 ms) : 0, 60595
Debugger [candidate] (61.085 ms) : 0, 61085
Remote Config [baseline] (605.598 µs) : 0, 606
Remote Config [candidate] (598.971 µs) : 0, 599
Telemetry [baseline] (9.974 ms) : 0, 9974
Telemetry [candidate] (8.43 ms) : 0, 8430
Flare Poller [baseline] (9.125 ms) : 0, 9125
Flare Poller [candidate] (9.844 ms) : 0, 9844
section appsec
crashtracking [baseline] (1.232 ms) : 0, 1232
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (675.632 ms) : 0, 675632
BytebuddyAgent [candidate] (675.394 ms) : 0, 675394
AgentMeter [baseline] (12.195 ms) : 0, 12195
AgentMeter [candidate] (12.24 ms) : 0, 12240
GlobalTracer [baseline] (249.566 ms) : 0, 249566
GlobalTracer [candidate] (249.053 ms) : 0, 249053
IAST [baseline] (24.637 ms) : 0, 24637
IAST [candidate] (24.636 ms) : 0, 24636
AppSec [baseline] (185.056 ms) : 0, 185056
AppSec [candidate] (185.061 ms) : 0, 185061
Debugger [baseline] (64.6 ms) : 0, 64600
Debugger [candidate] (64.647 ms) : 0, 64647
Remote Config [baseline] (559.433 µs) : 0, 559
Remote Config [candidate] (602.596 µs) : 0, 603
Telemetry [baseline] (7.758 ms) : 0, 7758
Telemetry [candidate] (7.835 ms) : 0, 7835
Flare Poller [baseline] (8.566 ms) : 0, 8566
Flare Poller [candidate] (7.821 ms) : 0, 7821
section iast
crashtracking [baseline] (1.226 ms) : 0, 1226
crashtracking [candidate] (1.237 ms) : 0, 1237
BytebuddyAgent [baseline] (825.029 ms) : 0, 825029
BytebuddyAgent [candidate] (829.294 ms) : 0, 829294
AgentMeter [baseline] (11.324 ms) : 0, 11324
AgentMeter [candidate] (11.424 ms) : 0, 11424
GlobalTracer [baseline] (238.355 ms) : 0, 238355
GlobalTracer [candidate] (239.415 ms) : 0, 239415
IAST [baseline] (28.267 ms) : 0, 28267
IAST [candidate] (28.322 ms) : 0, 28322
AppSec [baseline] (31.655 ms) : 0, 31655
AppSec [candidate] (31.808 ms) : 0, 31808
Debugger [baseline] (63.471 ms) : 0, 63471
Debugger [candidate] (64.021 ms) : 0, 64021
Remote Config [baseline] (531.124 µs) : 0, 531
Remote Config [candidate] (541.071 µs) : 0, 541
Telemetry [baseline] (7.987 ms) : 0, 7987
Telemetry [candidate] (8.099 ms) : 0, 8099
Flare Poller [baseline] (3.439 ms) : 0, 3439
Flare Poller [candidate] (3.534 ms) : 0, 3534
section profiling
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (699.053 ms) : 0, 699053
BytebuddyAgent [candidate] (696.85 ms) : 0, 696850
AgentMeter [baseline] (8.979 ms) : 0, 8979
AgentMeter [candidate] (8.976 ms) : 0, 8976
GlobalTracer [baseline] (210.799 ms) : 0, 210799
GlobalTracer [candidate] (209.364 ms) : 0, 209364
AppSec [baseline] (33.52 ms) : 0, 33520
AppSec [candidate] (32.693 ms) : 0, 32693
Debugger [baseline] (66.677 ms) : 0, 66677
Debugger [candidate] (65.486 ms) : 0, 65486
Remote Config [baseline] (603.305 µs) : 0, 603
Remote Config [candidate] (574.793 µs) : 0, 575
Telemetry [baseline] (8.181 ms) : 0, 8181
Telemetry [candidate] (8.116 ms) : 0, 8116
Flare Poller [baseline] (3.564 ms) : 0, 3564
Flare Poller [candidate] (3.549 ms) : 0, 3549
ProfilingAgent [baseline] (93.907 ms) : 0, 93907
ProfilingAgent [candidate] (93.674 ms) : 0, 93674
Profiling [baseline] (94.484 ms) : 0, 94484
Profiling [candidate] (94.224 ms) : 0, 94224
Loading

Load

Parameters

BaselineCandidate
baseline_or_candidatebaselinecandidate
git_branchmasterwahabi/llmobs-standalone-sampling-repro
git_commit_date17773868371777574060
git_commit_shae6cac64f4dbbf9
release_version1.62.0-SNAPSHOT~e6cac64dfd1.62.0-SNAPSHOT~f4dbbf9945
See matching parameters
BaselineCandidate
applicationinsecure-bankinsecure-bank
ci_job_date17775762841777576284
ci_job_id16475485371647548537
ci_pipeline_id110780158110780158
cpu_modelIntel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHzIntel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_versionLinux runner-zfyrx7zua-project-304-concurrent-0-ellqqdak 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/LinuxLinux runner-zfyrx7zua-project-304-concurrent-0-ellqqdak 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 15 unstable metrics.

scenarioΔ mean agg_http_req_duration_p50Δ mean agg_http_req_duration_p95Δ mean throughputcandidate mean agg_http_req_duration_p50candidate mean agg_http_req_duration_p95candidate mean throughputbaseline mean agg_http_req_duration_p50baseline mean agg_http_req_duration_p95baseline mean throughput
scenario:load:insecure-bank:iast:high_loadbetter
[-163.904µs; -63.185µs] or [-6.111%; -2.356%]		same
[-481.347µs; +57.072µs] or [-6.206%; +0.736%]		unstable
[-100.848op/s; +195.910op/s] or [-7.538%; +14.644%]		2.569ms7.543ms1385.344op/s2.682ms7.756ms1337.812op/s
scenario:load:petclinic:no_agent:high_loadbetter
[-1.763ms; -0.528ms] or [-9.512%; -2.846%]		unsure
[-2.516ms; -0.165ms] or [-8.256%; -0.541%]		unstable
[-13.854op/s; +40.354op/s] or [-5.643%; +16.437%]		17.393ms29.134ms258.750op/s18.538ms30.475ms245.500op/s
scenario:load:petclinic:iast:high_loadworse
[+0.585ms; +1.736ms] or [+3.327%; +9.861%]		unsure
[+0.407ms; +2.271ms] or [+1.426%; +7.949%]		unstable
[-40.418op/s; +13.668op/s] or [-15.530%; +5.252%]		18.760ms29.914ms246.875op/s17.599ms28.574ms260.250op/s
Request duration reports for insecure-bank
gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (1.321 ms) : 1309, 1333
. : milestone, 1321,
iast (3.424 ms) : 3378, 3470
. : milestone, 3424,
iast_FULL (6.008 ms) : 5947, 6068
. : milestone, 6008,
iast_GLOBAL (3.645 ms) : 3584, 3707
. : milestone, 3645,
profiling (2.152 ms) : 2131, 2174
. : milestone, 2152,
tracing (1.862 ms) : 1847, 1877
. : milestone, 1862,
section candidate
no_agent (1.25 ms) : 1239, 1261
. : milestone, 1250,
iast (3.304 ms) : 3259, 3349
. : milestone, 3304,
iast_FULL (6.01 ms) : 5950, 6070
. : milestone, 6010,
iast_GLOBAL (3.753 ms) : 3687, 3818
. : milestone, 3753,
profiling (2.164 ms) : 2145, 2183
. : milestone, 2164,
tracing (1.953 ms) : 1934, 1972
. : milestone, 1953,
Loading
  • baseline results
VariantRequest duration [CI 0.99]Δ no_agent
no_agent1.321 ms [1.309 ms, 1.333 ms]-
iast3.424 ms [3.378 ms, 3.47 ms]2.103 ms (159.2%)
iast_FULL6.008 ms [5.947 ms, 6.068 ms]4.687 ms (354.8%)
iast_GLOBAL3.645 ms [3.584 ms, 3.707 ms]2.324 ms (176.0%)
profiling2.152 ms [2.131 ms, 2.174 ms]831.487 µs (62.9%)
tracing1.862 ms [1.847 ms, 1.877 ms]540.795 µs (40.9%)
  • candidate results
VariantRequest duration [CI 0.99]Δ no_agent
no_agent1.25 ms [1.239 ms, 1.261 ms]-
iast3.304 ms [3.259 ms, 3.349 ms]2.054 ms (164.3%)
iast_FULL6.01 ms [5.95 ms, 6.07 ms]4.76 ms (380.8%)
iast_GLOBAL3.753 ms [3.687 ms, 3.818 ms]2.503 ms (200.2%)
profiling2.164 ms [2.145 ms, 2.183 ms]913.807 µs (73.1%)
tracing1.953 ms [1.934 ms, 1.972 ms]702.902 µs (56.2%)
Request duration reports for petclinic
gantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (19.012 ms) : 18822, 19202
. : milestone, 19012,
appsec (18.773 ms) : 18585, 18960
. : milestone, 18773,
code_origins (18.01 ms) : 17833, 18186
. : milestone, 18010,
iast (17.928 ms) : 17753, 18103
. : milestone, 17928,
profiling (18.503 ms) : 18322, 18684
. : milestone, 18503,
tracing (18.157 ms) : 17980, 18335
. : milestone, 18157,
section candidate
no_agent (18.037 ms) : 17855, 18219
. : milestone, 18037,
appsec (18.792 ms) : 18604, 18981
. : milestone, 18792,
code_origins (17.779 ms) : 17603, 17955
. : milestone, 17779,
iast (18.904 ms) : 18711, 19097
. : milestone, 18904,
profiling (18.355 ms) : 18172, 18537
. : milestone, 18355,
tracing (17.725 ms) : 17552, 17898
. : milestone, 17725,
Loading
  • baseline results
VariantRequest duration [CI 0.99]Δ no_agent
no_agent19.012 ms [18.822 ms, 19.202 ms]-
appsec18.773 ms [18.585 ms, 18.96 ms]-239.36 µs (-1.3%)
code_origins18.01 ms [17.833 ms, 18.186 ms]-1.003 ms (-5.3%)
iast17.928 ms [17.753 ms, 18.103 ms]-1.084 ms (-5.7%)
profiling18.503 ms [18.322 ms, 18.684 ms]-508.786 µs (-2.7%)
tracing18.157 ms [17.98 ms, 18.335 ms]-854.888 µs (-4.5%)
  • candidate results
VariantRequest duration [CI 0.99]Δ no_agent
no_agent18.037 ms [17.855 ms, 18.219 ms]-
appsec18.792 ms [18.604 ms, 18.981 ms]755.386 µs (4.2%)
code_origins17.779 ms [17.603 ms, 17.955 ms]-258.11 µs (-1.4%)
iast18.904 ms [18.711 ms, 19.097 ms]866.799 µs (4.8%)
profiling18.355 ms [18.172 ms, 18.537 ms]317.596 µs (1.8%)
tracing17.725 ms [17.552 ms, 17.898 ms]-311.666 µs (-1.7%)

Dacapo

Parameters

BaselineCandidate
baseline_or_candidatebaselinecandidate
git_branchmasterwahabi/llmobs-standalone-sampling-repro
git_commit_date17773868371777574060
git_commit_shae6cac64f4dbbf9
release_version1.62.0-SNAPSHOT~e6cac64dfd1.62.0-SNAPSHOT~f4dbbf9945
See matching parameters
BaselineCandidate
applicationbiojavabiojava
ci_job_date17775761021777576102
ci_job_id16475485431647548543
ci_pipeline_id110780158110780158
cpu_modelIntel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHzIntel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_versionLinux runner-zfyrx7zua-project-304-concurrent-0-idgqukp4 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/LinuxLinux runner-zfyrx7zua-project-304-concurrent-0-idgqukp4 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava
gantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (14.83 s) : 14830000, 14830000
. : milestone, 14830000,
appsec (15.151 s) : 15151000, 15151000
. : milestone, 15151000,
iast (18.167 s) : 18167000, 18167000
. : milestone, 18167000,
iast_GLOBAL (17.859 s) : 17859000, 17859000
. : milestone, 17859000,
profiling (14.756 s) : 14756000, 14756000
. : milestone, 14756000,
tracing (15.128 s) : 15128000, 15128000
. : milestone, 15128000,
section candidate
no_agent (15.549 s) : 15549000, 15549000
. : milestone, 15549000,
appsec (15.095 s) : 15095000, 15095000
. : milestone, 15095000,
iast (18.526 s) : 18526000, 18526000
. : milestone, 18526000,
iast_GLOBAL (18.081 s) : 18081000, 18081000
. : milestone, 18081000,
profiling (15.514 s) : 15514000, 15514000
. : milestone, 15514000,
tracing (14.751 s) : 14751000, 14751000
. : milestone, 14751000,
Loading
  • baseline results
VariantExecution Time [CI 0.99]Δ no_agent
no_agent14.83 s [14.83 s, 14.83 s]-
appsec15.151 s [15.151 s, 15.151 s]321.0 ms (2.2%)
iast18.167 s [18.167 s, 18.167 s]3.337 s (22.5%)
iast_GLOBAL17.859 s [17.859 s, 17.859 s]3.029 s (20.4%)
profiling14.756 s [14.756 s, 14.756 s]-74.0 ms (-0.5%)
tracing15.128 s [15.128 s, 15.128 s]298.0 ms (2.0%)
  • candidate results
VariantExecution Time [CI 0.99]Δ no_agent
no_agent15.549 s [15.549 s, 15.549 s]-
appsec15.095 s [15.095 s, 15.095 s]-454.0 ms (-2.9%)
iast18.526 s [18.526 s, 18.526 s]2.977 s (19.1%)
iast_GLOBAL18.081 s [18.081 s, 18.081 s]2.532 s (16.3%)
profiling15.514 s [15.514 s, 15.514 s]-35.0 ms (-0.2%)
tracing14.751 s [14.751 s, 14.751 s]-798.0 ms (-5.1%)
Execution time for tomcat
gantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f4dbbf9945, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (1.487 ms) : 1476, 1499
. : milestone, 1487,
appsec (3.805 ms) : 3583, 4026
. : milestone, 3805,
iast (2.274 ms) : 2204, 2343
. : milestone, 2274,
iast_GLOBAL (2.315 ms) : 2246, 2385
. : milestone, 2315,
profiling (2.089 ms) : 2035, 2144
. : milestone, 2089,
tracing (2.072 ms) : 2019, 2125
. : milestone, 2072,
section candidate
no_agent (1.488 ms) : 1476, 1499
. : milestone, 1488,
appsec (3.84 ms) : 3616, 4064
. : milestone, 3840,
iast (2.272 ms) : 2202, 2341
. : milestone, 2272,
iast_GLOBAL (2.321 ms) : 2251, 2391
. : milestone, 2321,
profiling (2.091 ms) : 2037, 2145
. : milestone, 2091,
tracing (2.074 ms) : 2021, 2127
. : milestone, 2074,
Loading
  • baseline results
VariantExecution Time [CI 0.99]Δ no_agent
no_agent1.487 ms [1.476 ms, 1.499 ms]-
appsec3.805 ms [3.583 ms, 4.026 ms]2.318 ms (155.9%)
iast2.274 ms [2.204 ms, 2.343 ms]786.632 µs (52.9%)
iast_GLOBAL2.315 ms [2.246 ms, 2.385 ms]828.373 µs (55.7%)
profiling2.089 ms [2.035 ms, 2.144 ms]601.96 µs (40.5%)
tracing2.072 ms [2.019 ms, 2.125 ms]584.896 µs (39.3%)
  • candidate results
VariantExecution Time [CI 0.99]Δ no_agent
no_agent1.488 ms [1.476 ms, 1.499 ms]-
appsec3.84 ms [3.616 ms, 4.064 ms]2.352 ms (158.1%)
iast2.272 ms [2.202 ms, 2.341 ms]783.998 µs (52.7%)
iast_GLOBAL2.321 ms [2.251 ms, 2.391 ms]833.726 µs (56.0%)
profiling2.091 ms [2.037 ms, 2.145 ms]603.394 µs (40.6%)
tracing2.074 ms [2.021 ms, 2.127 ms]586.436 µs (39.4%)

@FouadWahabiFouadWahabi mentioned this pull request May 1, 2026
Open
5 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@FouadWahabi@matsumo-and