Add test reporting for Karate v2

[daniel-mohedano]

What Does This Do

• Adds a new karate-2.0 instrumentation module to implement test visibility for Karate 2.0+.
  • The implementation follows the same approach used for the original instrumentation in the karate-1.0 module.
  • Also implements skip-based features (ITR and disabled tests), as part of the tracing listener.

Motivation

Karate v2 is a complete ground-up rewrite of the framework. Because of this, the original karate-1.0 module cannot instrument it.

Additional Notes

A following PR will contain the changes regarding execution instrumentation for Karate v2.

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: SDTEST-3816

[cit-pr-commenter-54b7da]

Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	57.49	55.43	$\color{red}{\blacktriangle}$ +2.06	54.33	$\color{red}{\blacktriangle}$ +3.16	76/242
agentEvpProxy	55.39	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	21.62	22.09	$\color{green}{\blacktriangledown}$ -0.47	21.65	$\color{green}{\blacktriangledown}$ -0.03	39/126
agentless	18.55	18.82	$\color{green}{\blacktriangledown}$ -0.27	18.82	$\color{green}{\blacktriangledown}$ -0.27	38/123
agentlessCodeCoverage	19.83	20.39	$\color{green}{\blacktriangledown}$ -0.56	19.99	$\color{green}{\blacktriangledown}$ -0.16	38/122
agentlessLineCoverage	30.28	29.82	$\color{red}{\blacktriangle}$ +0.46	29.82	$\color{red}{\blacktriangle}$ +0.46	37/121

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	37.15	37.15	$\color{gray}{\bullet}$ 0.00	36.42	$\color{red}{\blacktriangle}$ +0.73	38/119
agentless	38.43	36.42	$\color{red}{\blacktriangle}$ +2.01	36.42	$\color{red}{\blacktriangle}$ +2.01	38/119
agentlessCodeCoverage	45.10	45.38	$\color{green}{\blacktriangledown}$ -0.28	44.48	$\color{red}{\blacktriangle}$ +0.62	38/119
agentlessLineCoverage	75.01	74.82	$\color{red}{\blacktriangle}$ +0.19	74.82	$\color{red}{\blacktriangle}$ +0.19	37/118

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	9.64	9.35	$\color{red}{\blacktriangle}$ +0.29	10.33	$\color{green}{\blacktriangledown}$ -0.69	38/118
agentless	10.33	10.33	$\color{gray}{\bullet}$ 0.00	10.13	$\color{red}{\blacktriangle}$ +0.20	38/118
agentlessCodeCoverage	16.75	17.03	$\color{green}{\blacktriangledown}$ -0.28	17.38	$\color{green}{\blacktriangledown}$ -0.63	37/116

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	10.88	9.54	$\color{red}{\blacktriangle}$ +1.34	9.54	$\color{red}{\blacktriangle}$ +1.34	36/40

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	13.24	12.87	$\color{red}{\blacktriangle}$ +0.37	13.13	$\color{red}{\blacktriangle}$ +0.11	34/117
agentless	12.65	11.88	$\color{red}{\blacktriangle}$ +0.77	12.12	$\color{red}{\blacktriangle}$ +0.53	34/116
agentlessCodeCoverage	15.67	15.11	$\color{red}{\blacktriangle}$ +0.56	15.11	$\color{red}{\blacktriangle}$ +0.56	34/116
agentlessLineCoverage	19.37	19.20	$\color{red}{\blacktriangle}$ +0.17	19.20	$\color{red}{\blacktriangle}$ +0.17	33/116

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	92.27	95.12	$\color{green}{\blacktriangledown}$ -2.85	93.23	$\color{green}{\blacktriangledown}$ -0.96	41/128
agentless	88.34	89.58	$\color{green}{\blacktriangledown}$ -1.24	89.58	$\color{green}{\blacktriangledown}$ -1.24	39/124
agentlessCodeCoverage	98.30	99.00	$\color{green}{\blacktriangledown}$ -0.70	99.00	$\color{green}{\blacktriangledown}$ -0.70	40/123
agentlessLineCoverage	99.44	99.00	$\color{red}{\blacktriangle}$ +0.44	99.00	$\color{red}{\blacktriangle}$ +0.44	38/121

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	86.69	87.80	$\color{green}{\blacktriangledown}$ -1.11	87.80	$\color{green}{\blacktriangledown}$ -1.11	39/126
agentless	80.90	81.05	$\color{green}{\blacktriangledown}$ -0.15	81.05	$\color{green}{\blacktriangledown}$ -0.15	39/125
agentlessCodeCoverage	96.48	97.04	$\color{green}{\blacktriangledown}$ -0.56	95.12	$\color{red}{\blacktriangle}$ +1.36	39/123
agentlessLineCoverage	114.34	111.62	$\color{red}{\blacktriangle}$ +2.72	111.62	$\color{red}{\blacktriangle}$ +2.72	38/122

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	20.52	19.20	$\color{red}{\blacktriangle}$ +1.32	19.20	$\color{red}{\blacktriangle}$ +1.32	40/130
agentless	18.01	19.20	$\color{green}{\blacktriangledown}$ -1.19	19.20	$\color{green}{\blacktriangledown}$ -1.19	39/127
agentlessCodeCoverage	22.86	22.99	$\color{green}{\blacktriangledown}$ -0.13	22.09	$\color{red}{\blacktriangle}$ +0.77	38/125
agentlessLineCoverage	44.09	44.48	$\color{green}{\blacktriangledown}$ -0.39	44.48	$\color{green}{\blacktriangledown}$ -0.39	37/131

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[cit-pr-commenter-54b7da]

Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.43	16.36	$\color{red}{\blacktriangle}$ +0.07	16.04	$\color{red}{\blacktriangle}$ +0.39	38/117
agentless	9.68	9.73	$\color{green}{\blacktriangledown}$ -0.05	9.73	$\color{green}{\blacktriangledown}$ -0.05	38/118
agentlessCodeCoverage	13.65	13.67	$\color{green}{\blacktriangledown}$ -0.02	13.40	$\color{red}{\blacktriangle}$ +0.25	39/117
agentlessLineCoverage	32.89	32.95	$\color{green}{\blacktriangledown}$ -0.06	32.95	$\color{green}{\blacktriangledown}$ -0.06	37/116

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.75 s	14.60 s	[+0.2%; +1.9%] (maybe worse)
startup:insecure-bank:tracing:Agent	13.55 s	13.63 s	[-1.3%; +0.2%] (no difference)
startup:petclinic:appsec:Agent	16.86 s	16.71 s	[-0.1%; +1.9%] (no difference)
startup:petclinic:iast:Agent	16.92 s	16.83 s	[-0.2%; +1.3%] (no difference)
startup:petclinic:profiling:Agent	16.55 s	16.75 s	[-2.4%; +0.0%] (no difference)
startup:petclinic:sca:Agent	16.89 s	16.81 s	[-0.7%; +1.8%] (no difference)
startup:petclinic:tracing:Agent	15.63 s	15.96 s	[-6.4%; +2.2%] (no difference)

Commit:49a54e8d · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[cit-pr-commenter-54b7da]

Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-8.86	12.37	$\color{green}{\blacktriangledown}$ -21.23	13.67	$\color{green}{\blacktriangledown}$ -22.53	36/123
agentless	18.79	13.40	$\color{red}{\blacktriangle}$ +5.39	16.04	$\color{red}{\blacktriangle}$ +2.75	36/122
agentlessCodeCoverage	69.02	77.88	$\color{green}{\blacktriangledown}$ -8.86	86.07	$\color{green}{\blacktriangledown}$ -17.05	36/122
agentlessLineCoverage	139.85	130.99	$\color{red}{\blacktriangle}$ +8.86	136.34	$\color{red}{\blacktriangle}$ +3.51	35/121

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

[daniel-mohedano]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:2599ffbfcf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[daniel-mohedano]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:e459f4bc8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[datadog-official]

More details

The karate-2.0 instrumentation is behaviourally correct for its stated scope (basic test reporting + ITR skipping). The most non-obvious risk — that SCENARIO_EXIT fires even when the listener vetoes SCENARIO_ENTER — was confirmed safe: Karate v2 populates the result with skipped-step entries, so afterScenario does not spuriously call onTestSkip, and the unconditional onTestFinish call is a no-op because onTestIgnore already removed the test from inProgressTests. Tag names are stored without the @ prefix, confirming the datadog_itr_unskippable guard works correctly.

Was this helpful? React 👍 or 👎

_{📊 Validated against 4 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit e459f4b · What is Autotest? · Any feedback? Reach out in #autotest}

[daniel-mohedano]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-07-14 15:12:08 UTC ℹ️ Start processing command /merge

---

2026-07-14 15:12:14 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

---

2026-07-14 16:48:08 UTC ℹ️ MergeQueue: This merge request was merged