(fix): gate FrameType::isRawPointer on HotSpot

[jbachorik]

What does this PR do?:
Gates FrameType::isRawPointer()'s bit-30 raw-pointer bci encoding to HotSpot only, and fixes frame_ut.cpp to test the hotspot/non-hotspot cases separately now that the check is VM-gated.

Motivation:
The raw-pointer bci encoding (bit 30) only ever means "jmethodID slot replaced by a VMMethod*" on HotSpot. On OpenJ9, an unencoded, VM-supplied bci can coincidentally have bit 30 set, which was misread as our raw-pointer encoding and routed into JVMSupport::resolve(), which asserts/aborts for non-HotSpot VMs. Reproduced on Semeru 8.0.462/OpenJ9 via GetLineNumberTableLeakTest.

Additional Notes:
None.

How to test the change?:
frame_ut.cpp's TrueWhenBit30IsSet test now covers both the HotSpot and non-HotSpot cases explicitly.

For Datadog employees:

• If this PR touches code that signs or publishes builds or packages, or handles
  credentials of any kind, I've requested a security review (run the dd:platform-security-review
  skill, or file a request via the PSEC review form).
  bewaire also runs automatically on every PR.
• This PR doesn't touch any of that.
• JIRA: PROF-15397 [PROF-15364]

Unsure? Have a question? Request a review!

[dd-octo-sts]

CI Test Results

Run:#29326785663 | Commit:fd9914e | Duration: 17m 13s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK	glibc-aarch64/debug	glibc-amd64/debug	musl-aarch64/debug	musl-amd64/debug
8	-	✅	-	-
8-ibm	-	✅	-	-
8-j9	✅	✅	-	-
8-librca	-	-	✅	✅
8-orcl	-	✅	-	-
11	-	✅	-	-
11-j9	✅	✅	-	-
11-librca	-	-	✅	✅
17	✅	✅	-	-
17-graal	✅	✅	-	-
17-j9	✅	✅	-	-
17-librca	-	-	✅	✅
21	✅	✅	-	-
21-graal	✅	✅	-	-
21-librca	-	-	✅	✅
25	✅	✅	-	-
25-graal	✅	✅	-	-
25-librca	-	-	✅	✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0

---

Updated: 2026-07-14 11:11:39 UTC

[dd-octo-sts]

Benchmark Results (commit 552da8c)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124497089 Commit: 552da8c8bbdcbcbc3ff923ba7c13e7540aafe468

⚠️ Significant outliers

• 🔴 fj-kmeans (JDK 21): runtime +4% (2725→2834 ms)
• 🔴 future-genetic (JDK 21): runtime +3.4% (2029→2098 ms)

Runtime details (per benchmark × JDK)

Benchmark	JDK	Latest	Dev	Δ (dev vs latest)	Issues L/D
akka-uct	21	✅ 10311 ms (21 iters)	✅ 10141 ms (21 iters)	≈ -1.6% (±10.3%)	— / —
finagle-chirper	25	✅ 5418 ms (36 iters)	✅ 5553 ms (35 iters)	≈ +2.5% (±24.8%)	⚠️ W:3 / ⚠️ W:3
fj-kmeans	21	✅ 2725 ms (68 iters)	✅ 2834 ms (66 iters)	🔴 +4%	— / —
future-genetic	21	✅ 2029 ms (91 iters)	✅ 2098 ms (88 iters)	🔴 +3.4%	— / —
future-genetic	25	✅ 2032 ms (91 iters)	✅ 1998 ms (93 iters)	≈ -1.7% (±2.8%)	— / —
naive-bayes	25	✅ 1021 ms (168 iters)	✅ 1013 ms (168 iters)	≈ -0.8% (±31.6%)	— / —
reactors	21	✅ 16371 ms (15 iters)	✅ 15634 ms (15 iters)	≈ -4.5% (±7.1%)	— / —
reactors	25	✅ 18447 ms (15 iters)	✅ 18524 ms (15 iters)	≈ +0.4% (±4.4%)	— / —

Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark	JDK	Dropped rec	Dropped jvmti	Dropped trace	Skipped WC	AGCT fail	Unwind fail
akka-uct	21	✅ / ✅	✅ / ✅	3 / 1	2110 / 1969	✅ / ✅	✅ / ✅
akka-uct	25	✅ / ✅	✅ / ✅	1 / ✅	2300 / 2239	✅ / ✅	✅ / ✅
finagle-chirper	25	✅ / ✅	✅ / ✅	1 / ✅	8440 / 8369	✅ / ✅	✅ / ✅
fj-kmeans	21	✅ / ✅	✅ / ✅	1 / 2	1270 / 1248	✅ / ✅	✅ / ✅
fj-kmeans	25	✅ / ✅	✅ / ✅	3 / ✅	1253 / ✅	✅ / ✅	✅ / ✅
future-genetic	21	✅ / ✅	✅ / ✅	2 / ✅	2949 / 3050	✅ / ✅	✅ / ✅
future-genetic	25	✅ / ✅	✅ / ✅	2 / 1	2803 / 2935	✅ / ✅	✅ / ✅
naive-bayes	25	✅ / ✅	✅ / ✅	4 / 5	3471 / 3507	✅ / ✅	✅ / ✅
reactors	21	✅ / ✅	✅ / ✅	2 / ✅	1624 / 1635	✅ / ✅	✅ / ✅
reactors	25	✅ / ✅	✅ / ✅	✅ / 3	1928 / 1862	✅ / ✅	✅ / ✅

[Copilot]

Pull request overview

This PR fixes an OpenJ9-specific misclassification by making FrameType::isRawPointer() treat bit-30 as the raw-pointer encoding only when running on HotSpot, preventing non-HotSpot BCIs with bit 30 set from being routed into HotSpot-only resolution paths.

Changes:

• Gate FrameType::isRawPointer(int) on VM::isHotspot() in frame.h.
• Update frame_ut.cpp to split the previous “bit 30 set” assertion into explicit HotSpot vs non-HotSpot tests (with a test-only accessor to toggle VM::_hotspot).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
ddprof-lib/src/main/cpp/frame.h	Gates raw-pointer detection on VM::isHotspot() to avoid OpenJ9 false-positives.
ddprof-lib/src/test/cpp/frame_ut.cpp	Adjusts unit tests to validate both HotSpot and non-HotSpot behavior now that detection is VM-gated.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.