Migrate OpenTelemetry smoke test to JUnit by PerfectSlayer · Pull Request #12127 · DataDog/dd-trace-java

PerfectSlayer · 2026-08-03T13:26:09Z

What Does This Do

This PR migrates the OpenTelemetry smoke tests to the new JUnit smoke test framework.

Motivation

The new tests checks the whole trace structures and span values, in addition to trace payload validity and invariants.

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1247

datadog-datadog-prod-us1-2

More details

The JUnit migration preserves the OpenTelemetry smoke application launch and validates its one annotated trace plus ten sequential manual traces with the new backend assertions. No diff-only behavioral regression was confirmed; no additional tests recommended because the existing application shape is deterministic and the attempted module execution was blocked by unavailable Gradle/JDK tooling.

Was this helpful? React 👍 or 👎

_{📊 Validated against 3 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 9eece4a · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:9eece4ad18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dd-octo-sts · 2026-08-03T13:51:01Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.89 s	14.76 s	[+0.0%; +1.8%] (maybe worse)
startup:insecure-bank:tracing:Agent	13.60 s	13.74 s	[-1.7%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent	16.40 s	16.64 s	[-5.7%; +2.8%] (no difference)
startup:petclinic:iast:Agent	16.90 s	16.88 s	[-0.7%; +1.0%] (no difference)
startup:petclinic:profiling:Agent	16.71 s	16.88 s	[-2.3%; +0.2%] (no difference)
startup:petclinic:sca:Agent	16.24 s	16.56 s	[-6.4%; +2.4%] (no difference)
startup:petclinic:tracing:Agent	16.04 s	16.09 s	[-1.2%; +0.6%] (no difference)

Commit:bb1762be · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

datadog-datadog-prod-us1-2 · 2026-08-04T11:47:56Z

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 58.15% (-0.47%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: bb1762b | Docs | View more details | Give us feedback!}

datadog-datadog-prod-us1-2 · 2026-08-05T16:19:15Z

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

OpenTelemetrySmokeTest assumed trace delivery order when validating the annotated and manual OpenTelemetry spans. It now uses UNORDERED matching so the test validates all expected traces without depending on their arrival order.

Commit fix to this PR

View in Datadog | Reviewed commit 36e7755 · Any feedback? Reach out in #deveng-pr-agent

sarahchen6

👍

AlexeyKuznetsov-DD · 2026-08-14T00:11:22Z

+ trace(span().root().operationName("Application.annotatedSpan")),
+ trace(span().root().resourceName("span-0")),
+ trace(span().root().resourceName("span-1")),
+ trace(span().root().resourceName("span-2")),
+ trace(span().root().resourceName("span-3")),
+ trace(span().root().resourceName("span-4")),
+ trace(span().root().resourceName("span-5")),
+ trace(span().root().resourceName("span-6")),
+ trace(span().root().resourceName("span-7")),
+ trace(span().root().resourceName("span-8")),
+ trace(span().root().resourceName("span-9")));


nit: Looks like a copy-paste
Can we think of some more elegant checks?

Indeed, in the groovy equivalent it was waitForTraceCount(11), but this method's use is a tad pervasive, I found 152 callsites.
Also there's a Traces::waitForTraceCount method, shouldn't it be used ?

Can we think of some more elegant checks?
This come from the trace structure that is useless and not representing anything meanful.
The goal would be more to exercice the OTel API to build meaningful traces, which should give more elegant assert rules.
But it feels out of scope for this PR. The PR now checks the resource names, which is still a win compared to tho original trace count only.
Also there's a Traces::waitForTraceCount method, shouldn't it be used ?
The goal is to move to stricter trace structure testing. Trace count won't check for the format.
I expect to refine the test app behavior to something more meaningful so I only have basic test asserts for now.

AlexeyKuznetsov-DD · 2026-08-14T00:11:59Z

+com.fasterxml.jackson.core:jackson-annotations:2.10.3=testCompileClasspath,testRuntimeClasspath
+com.github.docker-java:docker-java-api:3.4.2=testCompileClasspath,testRuntimeClasspath
+com.github.docker-java:docker-java-transport-zerodep:3.4.2=testCompileClasspath,testRuntimeClasspath
+com.github.docker-java:docker-java-transport:3.4.2=testCompileClasspath,testRuntimeClasspath


Just curious why lock file changed?

This might come from the addition of test containers. I regenerated and the content seems up-to-date.

bric3

Looks good to me, however I think the comment on assertTrace and the list of spans deserves some attention.

bric3 · 2026-08-14T12:49:33Z

+ trace(span().root().operationName("Application.annotatedSpan")),
+ trace(span().root().resourceName("span-0")),
+ trace(span().root().resourceName("span-1")),
+ trace(span().root().resourceName("span-2")),
+ trace(span().root().resourceName("span-3")),
+ trace(span().root().resourceName("span-4")),
+ trace(span().root().resourceName("span-5")),
+ trace(span().root().resourceName("span-6")),
+ trace(span().root().resourceName("span-7")),
+ trace(span().root().resourceName("span-8")),
+ trace(span().root().resourceName("span-9")));


Indeed, in the groovy equivalent it was waitForTraceCount(11), but this method's use is a tad pervasive, I found 152 callsites.
Also there's a Traces::waitForTraceCount method, shouldn't it be used ?

datadog-datadog-prod-us1-2

More details

The JUnit test keeps the prior check for 11 traces and exit value 0. It also checks each trace and span with the new smoke test framework.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 8808628 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-datadog-prod-us1-2

More details

The JUnit migration preserves the 11 expected OpenTelemetry traces and the zero exit code check. The shared smoke-test framework has no reportable static correctness issue.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit bb1762b · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

PerfectSlayer added the type: feature Enhancements and improvements label Aug 3, 2026

PerfectSlayer requested a review from a team as a code owner August 3, 2026 13:26

PerfectSlayer added the tag: no release notes Changes to exclude from release notes label Aug 3, 2026

PerfectSlayer requested review from mtoffl01 and removed request for a teamAugust 3, 2026 13:26

PerfectSlayer added the inst: opentelemetry OpenTelemetry instrumentation label Aug 3, 2026

PerfectSlayer mentioned this pull request Aug 3, 2026
Improving smoke test capabilities #12051
Draft

datadog-datadog-prod-us1-2Bot reviewed Aug 3, 2026
View reviewed changes

chatgpt-codex-connectorBot reviewed Aug 3, 2026
View reviewed changes

Comment threaddd-smoke-tests/opentelemetry/src/test/java/datadog/smoketest/OpenTelemetrySmokeTest.java Outdated

PerfectSlayer requested a review from a team as a code owner August 4, 2026 11:30

PerfectSlayer requested a review from mcculls August 4, 2026 11:30

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch 2 times, most recently from e3bc1dc to f9175c8Compare August 4, 2026 11:32

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from f9175c8 to 36e7755Compare August 5, 2026 15:48

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch 2 times, most recently from 13ba721 to 97b671fCompare August 6, 2026 11:47

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 97b671f to 54741a2Compare August 7, 2026 08:13

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 54741a2 to 5eaac9aCompare August 11, 2026 09:55

PerfectSlayer added tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Aug 12, 2026

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 5eaac9a to 7f045dfCompare August 13, 2026 12:33

sarahchen6 approved these changes Aug 13, 2026
View reviewed changes

AlexeyKuznetsov-DD reviewed Aug 14, 2026
View reviewed changes

bric3 approved these changes Aug 14, 2026
View reviewed changes

PerfectSlayer requested a review from a team as a code owner August 20, 2026 16:15

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 7f045df to 8808628Compare August 20, 2026 16:15

datadog-datadog-prod-us1-2Bot reviewed Aug 20, 2026
View reviewed changes

bm1549 removed the request for review from mtoffl01 August 20, 2026 20:35

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 8808628 to b88ff9eCompare August 21, 2026 06:24

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from b88ff9e to 921c1ccCompare August 21, 2026 08:10

feat(testing): Add smoke test JUnit framework
5922fd3

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 921c1cc to 24d9a52Compare August 21, 2026 08:48

feat(opentelemetry): Migrate smoke test to JUnit
bb1762b

PerfectSlayer force-pushed the bbujon/smoke-tests-opentelemetry branch from 24d9a52 to bb1762bCompare August 21, 2026 11:27

Base automatically changed from bbujon/smoke-tests-junit-framework to masterAugust 21, 2026 12:22

datadog-datadog-prod-us1-2Bot reviewed Aug 21, 2026
View reviewed changes

Uh oh!

Conversation

PerfectSlayer commented Aug 3, 2026• edited by atlassian BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1-2Bot left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

dd-octo-stsBot commented Aug 3, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

datadog-datadog-prod-us1-2Bot commented Aug 4, 2026• edited by datadog-official BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-datadog-prod-us1-2Bot commented Aug 5, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bits has a CI fix ready

Uh oh!

sarahchen6 left a comment

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DDAug 14, 2026

Choose a reason for hiding this comment

Uh oh!

bric3Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

PerfectSlayerAug 21, 2026

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DDAug 14, 2026

Choose a reason for hiding this comment

Uh oh!

PerfectSlayerAug 21, 2026

Choose a reason for hiding this comment

Uh oh!

bric3 left a comment

Choose a reason for hiding this comment

Uh oh!

bric3Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PerfectSlayer commented Aug 3, 2026•
edited by atlassian Bot
Loading

dd-octo-stsBot commented Aug 3, 2026•
edited
Loading

datadog-datadog-prod-us1-2Bot commented Aug 4, 2026•
edited by datadog-official Bot
Loading

datadog-datadog-prod-us1-2Bot commented Aug 5, 2026•
edited
Loading