Use ThreadLocal to setup protection longjmp buffer and fix jmp_buf chaining by zhengyu123 · Pull Request #615 · DataDog/java-profiler

zhengyu123 · 2026-06-24T20:23:52Z

What does this PR do?:
This PR intents to resolve some of mysterious crashes seen in later releases, e.g. PROF-15206, although, it might not actually solve this particular problem, but I believe it should have positive impact - Cloude and I believe that overwriting JavaThread's exception field is dangerous, in fact, overwriting any JVM data structures are dangerous, because the writes can result in races and readers may see wrong values, that can lead to crashes.

Before introducing Thread Local Handshake in JVM, thread object is usually only read by 'current` thread, so it is relative safe to alter, then restore values inside signal handler, but not any more - a thread may be able to read other thread's states before it decides how to proceed the handshake, so it may read wrong values.

This PR:
Avoid altering JavaThread for jmp_buf
Uses a thread local for storing longjmp buffer and eliminates the dependency on overwriting JavaThread's exception field, so

Avoiding other thread reads wrong value of the exception field.
longjmp now protects all threads, not just JavaThread - should improve stability.

Chaining jmp_buf
A none signal based sampler can be interrupted by signal based sampler. Therefore, there may be multiple walkVM() calls on stack, each of them sets up a protection jmp_buf for stack walk, the jmp_bufs need to be chained to ensure to jump back to expected location.

Motivation:
Improve profiler's stability.

Additional Notes:
In additional, exception field only exists in JavaThread, but detection code in Profiler is somewhat questionable, especially, for the subclasses of JavaThread

How to test the change?:

CI tests
New unit tests

For Datadog employees:

If this PR touches code that signs or publishes builds or packages, or handles
credentials of any kind, I've requested a security review (run the dd:platform-security-review
skill, or file a request via the PSEC review form).
bewaire also runs automatically on every PR.
This PR doesn't touch any of that.
JIRA: PROF-15206

Unsure? Have a question? Request a review!

datadog-datadog-prod-us1-2 · 2026-06-24T20:33:04Z

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 6 Pipeline jobs failed

Code Quality Checks | codeql (cpp)
Code Quality Checks | codeql (java)
DataDog/java-profiler | reliability-chaos-aarch64: [profiler+tracer, tcmalloc, 21.0.3-tem]
View all 6 failed jobs.

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: c71118d | Docs | Datadog PR Page | Give us feedback!}

dd-octo-sts · 2026-06-24T20:41:37Z

CI Test Results

Run:#28827871146 | Commit:f81588b | Duration: 13m 55s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK	glibc-aarch64/debug	glibc-amd64/debug	musl-aarch64/debug	musl-amd64/debug
8	-	✅	-	-
8-ibm	-	✅	-	-
8-j9	✅	✅	-	-
8-librca	-	-	✅	✅
8-orcl	-	✅	-	-
11	-	✅	-	-
11-j9	✅	✅	-	-
11-librca	-	-	✅	✅
17	✅	✅	-	-
17-graal	✅	✅	-	-
17-j9	✅	✅	-	-
17-librca	-	-	✅	✅
21	✅	✅	-	-
21-graal	✅	✅	-	-
21-librca	-	-	✅	✅
25	✅	✅	-	-
25-graal	✅	✅	-	-
25-librca	-	-	✅	✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0

Updated: 2026-07-06 22:52:10 UTC

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

dd-octo-sts · 2026-06-25T01:26:09Z

Reliability & Chaos Results

❌ 2 failure(s) detected Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/123082923

❌ chaos: profiler tcmalloc amd64 21 0 3 temXchaos

Chaos harness crashed (RC=124)

❌ chaos: profiler tracer tcmalloc aarch64 25 0 3 temXchaos

Chaos harness crashed (RC=124)

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.

…text

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…ofiler into zgu/objmonitor_deflation

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 9 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

dd-octo-sts · 2026-07-03T17:31:47Z

Benchmark Results (commit `e4b9719`)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/122633659 Commit: e4b97194a1bd758dfffcb77ab669046e94fe63bc

⚠️ Significant outliers

🟢 fj-kmeans (JDK 21): runtime -5% (2875→2730 ms)
🔴 fj-kmeans (JDK 25): runtime +7.9% (2633→2842 ms)
⚠️reactors (JDK 21): wallclock unwinds skipped — latest 0 → dev 1597 (new)

Runtime details (per benchmark × JDK)

Benchmark	JDK	Latest	Dev	Δ (dev vs latest)	Issues L/D
akka-uct	21	✅ 10389 ms (7 iters)	✅ 10329 ms (7 iters)	≈ -0.6% (±20.8%)	— / —
akka-uct	25	✅ 8966 ms (8 iters)	✅ 8809 ms (8 iters)	≈ -1.8% (±20.3%)	— / —
finagle-chirper	21	✅ 6023 ms (11 iters)	✅ 6002 ms (11 iters)	≈ -0.3% (±44.9%)	⚠️ W:1 / ⚠️ W:1
finagle-chirper	25	✅ 5457 ms (12 iters)	✅ 5438 ms (12 iters)	≈ -0.3% (±43.4%)	⚠️ W:1 / ⚠️ W:1
fj-kmeans	21	✅ 2875 ms (22 iters)	✅ 2730 ms (23 iters)	🟢 -5%	— / —
fj-kmeans	25	✅ 2633 ms (24 iters)	✅ 2842 ms (22 iters)	🔴 +7.9%	— / —
future-genetic	21	✅ 2133 ms (29 iters)	✅ 2080 ms (30 iters)	≈ -2.5% (±4.2%)	— / —
future-genetic	25	✅ 2070 ms (30 iters)	✅ 2141 ms (29 iters)	≈ +3.4% (±4.3%)	— / —
naive-bayes	21	✅ 1221 ms (46 iters)	✅ 1277 ms (45 iters)	≈ +4.6% (±57.5%)	— / —
naive-bayes	25	✅ 1034 ms (55 iters)	✅ 1018 ms (56 iters)	≈ -1.5% (±55.8%)	— / —
reactors	21	✅ 16041 ms (5 iters)	✅ 15714 ms (5 iters)	≈ -2% (±17.4%)	— / —
reactors	25	✅ 17707 ms (5 iters)	✅ 17823 ms (5 iters)	≈ +0.7% (±11.5%)	— / —

Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark	JDK	Dropped rec	Dropped jvmti	Dropped trace	Skipped WC	AGCT fail	Unwind fail
akka-uct	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅
akka-uct	25	✅ / ✅	✅ / ✅	1 / 4	2466 / 2284	✅ / ✅	✅ / ✅
finagle-chirper	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅
finagle-chirper	25	✅ / ✅	✅ / ✅	1 / 1	8316 / 8172	✅ / ✅	✅ / ✅
fj-kmeans	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅
fj-kmeans	25	✅ / ✅	✅ / ✅	2 / 4	1298 / 1271	✅ / ✅	✅ / ✅
future-genetic	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅
future-genetic	25	✅ / ✅	✅ / ✅	3 / 1	2914 / 2977	✅ / ✅	✅ / ✅
naive-bayes	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅
naive-bayes	25	✅ / ✅	✅ / ✅	4 / 1	3478 / 3494	✅ / ✅	✅ / ✅
reactors	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / 1597	✅ / ✅	✅ / ✅
reactors	25	✅ / ✅	✅ / ✅	✅ / 2	1765 / 1822	✅ / ✅	✅ / ✅

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:35d3ad2be2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dd-octo-sts · 2026-07-06T13:02:10Z

Benchmark Results (commit `4b571c4`)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/122986649 Commit: 4b571c44baa590473687e46b003b9e00256421da

⚠️ Significant outliers

🟢 fj-kmeans (JDK 21): runtime -5% (2872→2727 ms)
🔴 future-genetic (JDK 25): runtime +5.6% (2018→2131 ms)
🔴 reactors (JDK 21): runtime +11% (16572→18390 ms)
⚠️fj-kmeans (JDK 21): wallclock unwinds skipped — latest 0 → dev 1268 (new)

Runtime details (per benchmark × JDK)

Benchmark	JDK	Latest	Dev	Δ (dev vs latest)	Issues L/D
akka-uct	21	✅ 10142 ms (7 iters)	✅ 10374 ms (7 iters)	≈ +2.3% (±21.9%)	— / —
akka-uct	25	✅ 8697 ms (8 iters)	✅ 8704 ms (8 iters)	≈ +0.1% (±16.9%)	— / —
finagle-chirper	21	✅ 6027 ms (11 iters)	✅ 5956 ms (11 iters)	≈ -1.2% (±45.3%)	⚠️ W:1 / ⚠️ W:1
finagle-chirper	25	✅ 5449 ms (12 iters)	✅ 5449 ms (12 iters)	≈ 0% (±43.5%)	⚠️ W:1 / ⚠️ W:1
fj-kmeans	21	✅ 2872 ms (22 iters)	✅ 2727 ms (23 iters)	🟢 -5%	— / —
fj-kmeans	25	✅ 2824 ms (22 iters)	✅ 2791 ms (22 iters)	≈ -1.2% (±4.6%)	— / —
future-genetic	21	✅ 2034 ms (30 iters)	✅ 2088 ms (30 iters)	≈ +2.7% (±4.3%)	— / —
future-genetic	25	✅ 2018 ms (31 iters)	✅ 2131 ms (29 iters)	🔴 +5.6%	— / —
naive-bayes	21	✅ 1234 ms (46 iters)	✅ 1327 ms (43 iters)	≈ +7.5% (±58.8%)	— / —
naive-bayes	25	✅ 1017 ms (56 iters)	✅ 1017 ms (56 iters)	≈ 0% (±55.2%)	— / —
reactors	21	✅ 16572 ms (5 iters)	✅ 18390 ms (5 iters)	🔴 +11%	— / —
reactors	25	✅ 18269 ms (5 iters)	✅ 18396 ms (5 iters)	≈ +0.7% (±6%)	— / —

Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark	JDK	Dropped rec	Dropped jvmti	Dropped trace	Skipped WC	AGCT fail	Unwind fail
akka-uct	21	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅	✅ / ✅
akka-uct	25	✅ / ✅	✅ / ✅	1 / 2	2289 / 2053	✅ / ✅	✅ / ✅
finagle-chirper	21	✅ / ✅	✅ / ✅	2 / ✅	8650 / ✅	✅ / ✅	✅ / ✅
finagle-chirper	25	✅ / ✅	✅ / ✅	✅ / ✅	8159 / 8478	✅ / ✅	✅ / ✅
fj-kmeans	21	✅ / ✅	✅ / ✅	✅ / 5	✅ / 1268	✅ / ✅	✅ / ✅
fj-kmeans	25	✅ / ✅	✅ / ✅	7 / 4	1275 / 1265	✅ / ✅	✅ / ✅
future-genetic	21	✅ / ✅	✅ / ✅	1 / ✅	2953 / ✅	✅ / ✅	✅ / ✅
future-genetic	25	✅ / ✅	✅ / ✅	3 / ✅	2941 / ✅	✅ / ✅	✅ / ✅
naive-bayes	21	✅ / ✅	✅ / ✅	5 / ✅	3490 / ✅	✅ / ✅	✅ / ✅
naive-bayes	25	✅ / ✅	✅ / ✅	3 / 2	3489 / 3508	✅ / ✅	✅ / ✅
reactors	21	✅ / ✅	✅ / ✅	2 / 3	1692 / 1976	✅ / ✅	✅ / ✅
reactors	25	✅ / ✅	✅ / ✅	✅ / ✅	1875 / 1835	✅ / ✅	✅ / ✅

rkennke

Looks good to me. Thank you!

zhengyu123 · 2026-07-06T15:32:14Z

Looks good to me. Thank you!

@rkennke Thank you for reviewing.

jbachorik

A few minor/clarifying comments. Otherwise good to go.

…from deleted thread.h to threadLocalData.h PR #615 (origin/main) renamed thread.cpp to threadLocalData.cpp and deleted thread.h, absorbing ProfiledThread into threadLocalData.h. Our branch had modified thread.h to add per-thread JFR-inflight tracking. Resolution: - Delete thread.h (accept deletion from main) - Port _registry_head/_registry_lock static members to threadLocalData.h - Port _jfr_inflight/_registry_next instance members to threadLocalData.h - Port enterJfrInflight/exitJfrInflight/jfrInflight methods to threadLocalData.h - Port forEachRegistered/registryInsert/registryRemove declarations to threadLocalData.h - Update signalInflight.cpp to include threadLocalData.h instead of thread.h - threadLocalData.cpp already has implementation (correctly auto-merged)

v0
73facf0

zhengyu123 requested a review from Copilot June 24, 2026 20:23

Copilotstarted reviewing on behalf of zhengyu123 June 24, 2026 20:24 View session

CopilotAI reviewed Jun 24, 2026
View reviewed changes

zhengyu123and others added 4 commits June 24, 2026 16:58

Potential fix for pull request finding
ce7118f
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding
cb46ffa
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding
ba279d0
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding
c6936b5
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

zhengyu123 added 2 commits June 25, 2026 15:30

Merge branch 'main' into zgu/objmonitor_deflation
51ef013

v2
125fbb0

zhengyu123 requested a review from Copilot June 25, 2026 18:00

Copilotstarted reviewing on behalf of zhengyu123 June 25, 2026 18:01 View session

CopilotAI reviewed Jun 25, 2026
View reviewed changes

v3
1b077af

zhengyu123 requested a review from Copilot June 26, 2026 19:03

Copilotstarted reviewing on behalf of zhengyu123 June 26, 2026 19:03 View session

CopilotAI reviewed Jun 26, 2026
View reviewed changes

zhengyu123and others added 5 commits June 26, 2026 20:09

Remove dependency on java_thread.exception field to setup longjmp con…
bb3a192
…text

Potential fix for pull request finding
b929d3c
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

v4
b542dcd

Merge branch 'zgu/objmonitor_deflation' of github.com:DataDog/java-pr…
4a438cb
…ofiler into zgu/objmonitor_deflation

typo and etc.
b5a9432

zhengyu123 changed the title ~~wip:~~ wip: Use TheadLocal to setup protection longjmp bufferJun 29, 2026

zhengyu123 requested a review from Copilot June 29, 2026 13:30

Copilotstarted reviewing on behalf of zhengyu123 June 29, 2026 13:31 View session

CopilotAI reviewed Jun 29, 2026
View reviewed changes

Potential fix for pull request finding
a8a1119
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

zhengyu123 marked this pull request as ready for review July 6, 2026 12:08

Merge branch 'main' into zgu/objmonitor_deflation
35d3ad2

chatgpt-codex-connectorBot reviewed Jul 6, 2026
View reviewed changes

Comment threadddprof-test/src/test/java/com/datadoghq/profiler/cpu/MonitorDeflationThreadSafetyTest.java

Added copyright
4b571c4

rkennke approved these changes Jul 6, 2026
View reviewed changes

jbachorik reviewed Jul 6, 2026
View reviewed changes

Comment threadddprof-lib/src/main/cpp/jvmThread.cpp Outdated

jbachorik reviewed Jul 6, 2026
View reviewed changes

Comment threadddprof-lib/src/main/cpp/profiler.cpp Outdated

jbachorik reviewed Jul 6, 2026
View reviewed changes

Comment threadddprof-lib/src/main/cpp/vmEntry.cpp Outdated

jbachorik reviewed Jul 6, 2026
View reviewed changes

Comment threadddprof-test/src/test/java/com/datadoghq/profiler/cpu/MonitorDeflationThreadSafetyTest.java

jbachorik approved these changes Jul 6, 2026
View reviewed changes

Review comments
c71118d

gh-worker-dd-devflow-36fce6Bot added mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected and removed mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected labels Jul 6, 2026

zhengyu123 merged commit 3869f88 into mainJul 6, 2026
537 of 558 checks passed

zhengyu123 deleted the zgu/objmonitor_deflation branch July 6, 2026 23:14

github-actionsBot added this to the 1.47.0 milestone Jul 6, 2026

CopilotAI mentioned this pull request Jul 9, 2026
[PROF-15268] POC: per-thread JFR-inflight counter (follow-up to #614) #629
Draft
2 tasks

Uh oh!

Conversation

zhengyu123 commented Jun 24, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

datadog-datadog-prod-us1-2Bot commented Jun 24, 2026• edited by datadog-datadog-prod-us1 BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

dd-octo-stsBot commented Jun 24, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Test Results

Status Overview

Uh oh!

dd-octo-stsBot commented Jun 25, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reliability & Chaos Results

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dd-octo-stsBot commented Jul 3, 2026

Benchmark Results (commit e4b9719)

⚠️ Significant outliers

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

dd-octo-stsBot commented Jul 6, 2026

Benchmark Results (commit 4b571c4)

⚠️ Significant outliers

Uh oh!

rkennke left a comment

Choose a reason for hiding this comment

Uh oh!

zhengyu123 commented Jul 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jbachorik left a comment

zhengyu123 commented Jun 24, 2026•
edited
Loading

datadog-datadog-prod-us1-2Bot commented Jun 24, 2026•
edited by datadog-datadog-prod-us1 Bot
Loading

dd-octo-stsBot commented Jun 24, 2026•
edited
Loading

dd-octo-stsBot commented Jun 25, 2026•
edited
Loading

Benchmark Results (commit `e4b9719`)

Benchmark Results (commit `4b571c4`)