fix: own line number table memory to avoid UAF on class unload (PROF-14547) by jbachorik · Pull Request #511 · DataDog/java-profiler

jbachorik · 2026-05-06T23:25:38Z

What does this PR do?:

Fixes SIGSEGV in Profiler::processCallTraces (production stack: processCallTraces → Recording::writeStackTraces → Recording::writeCpool → Recording::finishChunk → FlightRecorder::dump → Profiler::dump). Detaches MethodInfo::_line_number_table->_ptr lifetime from JVMTI by copying the line number table into a malloc'd buffer at capture time inside Lookup::fillJavaMethodInfo and immediately calling jvmti->Deallocate on the JVMTI memory. ~SharedLineNumberTable now calls free() on the owned buffer.

Motivation:

Production SIGSEGV in processCallTraces+0xaa during JFR dump. Root cause: MethodInfo::_line_number_table->_ptr held a raw pointer to JVMTI-allocated memory returned by GetLineNumberTable. When the underlying class was unloaded, the JVM freed that memory; the next dump's writeStackTraces lambda called MethodInfo::getLineNumber(bci) on a dangling _ptr and crashed.

This is a peer manifestation of (PR #510), which crashes at the symmetric deallocation site (~SharedLineNumberTable → Deallocate). Both stem from the same dangling pointer; this fix addresses the lifetime invariant once and subsumes both crashes. PR #510 stays correct (becomes belt-and-suspenders) but is no longer required for the SIGSEGV fix.

Additional Notes:

Hot-path impact is once per first-time-encountered method per dump cycle (gated by !mi->_mark and mi->_key == 0). The new work is one malloc + memcpy of the line number table (typically 16–80 bytes) plus an immediate Deallocate — replaces the implicit JVMTI-side allocation that already happened. Not on the signal handler path (fillJavaMethodInfo runs on the JFR-writer thread under _state_lock + lockAll()).
Memory footprint is unchanged: same logical bytes per method, same lifetime, gated by the existing AGE_THRESHOLD=3 eviction.
malloc failure is handled silently on the read side (getLineNumber returns 0 when _line_number_table is null) and logged via TEST_LOG.
jvmti->Deallocate failure now logged via TEST_LOG (preserves the diagnostic that the destructor previously emitted).
Reviewed by 4 internal reviewer voices (concurrency, perf, error-paths, tests). Concerns addressed inline; deferred items: JMH benchmark for chunk-write latency on method-churn workloads (follow-up).

How to test the change?:

Added regression test WriteStackTracesAfterClassUnloadTest:

Generate a class with line-numbered methods via ASM.
Hammer its methods for ~300ms while CPU profiling at 1ms so traces land in call_trace_storage and a MethodInfo with _line_number_table is created.
Drop strong refs and System.gc() until the WeakReference clears (class actually unloaded); assumeTrue-skip if the JVM declined to unload within the deadline.
Dump 4 times in tight succession. Without the fix, getLineNumber dereferences the freed JVMTI memory → SIGSEGV. With the fix, _ptr is a malloc'd copy → safe.

Test set deliberately uses mcleanup=false to isolate the failure to the PROF-14547 read path; the PROF-14545 cleanup path is covered separately by CleanupAfterClassUnloadTest (PR #510). Best-effort under ASan/UBSan, which amplify the signal even when the freed region happens to read sensible bytes — limitations documented in the test javadoc.

Build verified locally: :ddprof-lib:assembleRelease and :ddprof-test:compileTestJava both pass on macOS arm64.

For Datadog employees:

If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
This PR doesn't touch any of that.
JIRA: PROF-14547

jbachorik · 2026-05-06T23:28:45Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:05a6f4cb00

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dd-octo-sts · 2026-05-06T23:57:10Z

CI Test Results

Run:#25755969382 | Commit:25f4cb2 | Duration: 31m 6s (longest job)

❌ 1 of 32 test jobs failed

Status Overview

JDK	glibc-aarch64/debug	glibc-amd64/debug	musl-aarch64/debug	musl-amd64/debug
8	-	✅	-	-
8-ibm	-	✅	-	-
8-j9	✅	✅	-	-
8-librca	-	-	✅	✅
8-orcl	-	✅	-	-
11	-	✅	-	-
11-j9	✅	✅	-	-
11-librca	-	-	❌	✅
17	✅	✅	-	-
17-graal	✅	✅	-	-
17-j9	✅	✅	-	-
17-librca	-	-	✅	✅
21	✅	✅	-	-
21-graal	✅	✅	-	-
21-librca	-	-	✅	✅
25	✅	✅	-	-
25-graal	✅	✅	-	-
25-librca	-	-	✅	✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Failed Tests

musl-aarch64/debug / 11-librca

Job:View logs

No detailed failure information available. Check the job logs.

Summary: Total: 32 | Passed: 31 | Failed: 1

Updated: 2026-05-12 19:58:39 UTC

jbachorik · 2026-05-07T07:14:35Z

@codex review

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:8b42102179

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

rkennke

Looks good overall, I only have a few minor comments.

rkennke

Thank you! Looks good, now!

…14547) Copy the JVMTI line number table into a malloc'd buffer at capture time and Deallocate the JVMTI memory immediately, detaching _ptr lifetime from class unload. Eliminates the SIGSEGV in MethodInfo::getLineNumber during writeStackTraces. Adds a regression test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

- Remove PROF-14545/PROF-14547 ticket references from comments in flightRecorder.cpp and flightRecorder.h - Fix samplingInterval metadata: use <=1 threshold to match shouldSample's interval<=1 condition (nativemem=1 also records every allocation) Co-Authored-By: muse <muse@noreply>

…adTest

- Remove dead `line_number_table = nullptr` (local var, no effect after Deallocate) - Extract AbstractDynamicClassTest base class; deduplicate generateClassBytecode, tempFile (now uses java.io.tmpdir), and IsolatedClassLoader from GetLineNumberTableLeakTest and WriteStackTracesAfterClassUnloadTest Co-Authored-By: muse <muse@noreply>

jbachorik · 2026-05-13T07:41:37Z

Subsumed into #510 — all commits merged into muse/sigsegv-in-recording.

jbachorik added the AI label May 6, 2026

jbachorik requested a review from Copilot May 6, 2026 23:28

Copilotstarted reviewing on behalf of jbachorik May 6, 2026 23:28 View session

This comment was marked as outdated.
Sign in to view

chatgpt-codex-connectorBot reviewed May 6, 2026
View reviewed changes

Comment threadddprof-lib/src/main/cpp/mallocTracer.cpp

jbachorik force-pushed the muse/fix-sigsegv-recording branch from 05a6f4c to 8b42102Compare May 7, 2026 07:10

jbachorik requested a review from Copilot May 7, 2026 07:12

Copilotstarted reviewing on behalf of jbachorik May 7, 2026 07:13 View session

CopilotAI reviewed May 7, 2026
View reviewed changes

Comment thread...-test/src/test/java/com/datadoghq/profiler/memleak/WriteStackTracesAfterClassUnloadTest.java Outdated

chatgpt-codex-connectorBot reviewed May 7, 2026
View reviewed changes

Comment thread...-test/src/test/java/com/datadoghq/profiler/memleak/WriteStackTracesAfterClassUnloadTest.java

jbachorik commented May 7, 2026
View reviewed changes

Comment threadddprof-lib/src/main/cpp/flightRecorder.cpp Outdated

jbachorik marked this pull request as ready for review May 7, 2026 20:25

jbachorik requested a review from a team as a code owner May 7, 2026 20:25

jbachorik requested a review from Copilot May 7, 2026 20:26

Copilotstarted reviewing on behalf of jbachorik May 7, 2026 20:27 View session

CopilotAI reviewed May 7, 2026
View reviewed changes

jbachorik added the AI: Author Signed-off label May 7, 2026

rkennke requested changes May 11, 2026
View reviewed changes

rkennke approved these changes May 11, 2026
View reviewed changes

jbachorik force-pushed the muse/fix-sigsegv-recording branch from 8eeddf9 to bb53d21Compare May 11, 2026 22:14

jbachorik changed the base branch from main to muse/sigsegv-in-recordingMay 11, 2026 22:14

jbachorik force-pushed the muse/fix-sigsegv-recording branch from bb53d21 to b485d32Compare May 12, 2026 00:33

jbachorik force-pushed the muse/sigsegv-in-recording branch from 1ad4c18 to d73f80cCompare May 12, 2026 05:56

jbachorik force-pushed the muse/fix-sigsegv-recording branch 2 times, most recently from 5df7be9 to c75a4a1Compare May 12, 2026 07:15

jbachorikand others added 2 commits May 12, 2026 21:02

Fix test robustness
71b51cc
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

jbachorikand others added 4 commits May 12, 2026 21:02

ci-watch: fix compileTestJava — profiler.resetThreadContext() (cycle 1)
0d0e44b

test: remove all JIRA ticket refs from WriteStackTracesAfterClassUnlo…
bf6ae85
…adTest

jbachorik force-pushed the muse/fix-sigsegv-recording branch from c75a4a1 to afdfea2Compare May 12, 2026 19:02

jbachorik closed this May 13, 2026

Uh oh!

Conversation

jbachorik commented May 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jbachorik commented May 6, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

dd-octo-stsBot commented May 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Test Results

Status Overview

Failed Tests

Uh oh!

jbachorik commented May 7, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

rkennke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rkennke left a comment

Choose a reason for hiding this comment

Uh oh!

jbachorik commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jbachorik commented May 6, 2026•
edited
Loading

dd-octo-stsBot commented May 6, 2026•
edited
Loading