profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

profiling-node: importing the package can hard-deadlock the process at startup on Node ≤22 (V8 bug 41497149 — eager v8::CpuProfiler creation at require time) #23178

Description

@vincentventalon

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/node

SDK Version

@sentry/nestjs 10.26.0, @sentry/profiling-node 10.45.0 (also reproduced with the 10.25.0 binary)

Framework Version

NestJS 10, Node.js 22.20.0 (V8 12.4.254.21), linux x64 glibc (Docker node:22.20-bookworm-slim)

Link to Sentry event

No event — the process freezes before any code runs.

Reproduction Example/SDK Setup

// first import of the app's main.tsimport{nodeProfilingIntegration}from'@sentry/profiling-node';

That import alone is enough to trigger the bug — in our case Sentry.init was not even called (no DSN configured), yet the process could freeze at startup.

Steps to Reproduce

  1. Run a large Node 22 app (NestJS monolith, thousands of modules) that imports @sentry/profiling-node after a sizable part of its module graph (we import it via a wrapper that is itself imported first, so ~the whole @sentry/nestjs + OTel graph is already compiled when the addon loads).
  2. Boot the app repeatedly under CPU/IO contention (in our case: 2-vCPU host, full docker-compose stack cold-starting simultaneously, page caches dropped — i.e. a normal container deploy).
  3. Roughly 1 boot in 3-4 freezes forever: zero output, listen port never bound, ~0 CPU, process never crashes. A process manager (pm2) keeps reporting it online.

The non-determinism is inherent: the deadlock only fires if V8 starts an incremental GC cycle exactly while the profiler walks already-compiled code (see below). Calm restarts of the same build essentially never hit it; contended cold boots hit it frequently.

Expected Result

Importing @sentry/profiling-node never freezes the process — ideally the v8::CpuProfiler is not created as a side effect of require() at all (lazily create it when profiling actually starts), especially when Sentry.init was never called.

Actual Result

Hard deadlock of the main thread inside the addon's N-API Init, on Node ≤ 22. gdb backtrace of the frozen main thread (all other threads idle in their normal wait loops — this is a same-thread self-deadlock):

#1 pthread_mutex_lock
#2 v8::internal::Isolate::AllowsCodeCompaction()
#3 v8::internal::MarkCompactCollector::StartCompaction(...)
#4 v8::internal::IncrementalMarking::StartMarkingMajor()
#6 v8::internal::Heap::StartIncrementalMarking(...)
#7 v8::internal::PagedSpaceAllocatorPolicy::EnsureAllocation(...)
#8 v8::internal::MainAllocator::AllocateRawSlow(...)
#11 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(...)
#12 v8::internal::String::CalculateLineEnds(...)
#13 v8::internal::Script::InitLineEndsInternal(...)
#14 v8::internal::Script::GetPositionInfo(...)
#16 v8::internal::SourcePosition::InliningStack(...)
#17 v8::internal::ProfilerListener::CodeCreateEvent(...)
#18 v8::internal::ExistingCodeLogger::LogExistingFunction(...)
#19 v8::internal::ExistingCodeLogger::LogCompiledFunctions(...)
#20 v8::internal::ProfilingScope::ProfilingScope(...)
#21 v8::internal::CpuProfiler::EnableLogging()
#22 v8::internal::CpuProfiler::CpuProfiler(...)
#23 v8::CpuProfiler::New(...)
#24 Init(napi_env__*, napi_value__*) ()
from .../node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-127.node
#25 napi_module_register_by_symbol(...)
#28 node::binding::DLOpen(...)

Mechanism (V8 side): v8::CpuProfiler::NewProfilingScope walks all compiled functions via Logger::CodeCreateEvent, which holds the (non-recursive) Logger mutex while dispatching. Initializing script line-ends inside that walk allocates; if that allocation starts incremental marking, MarkCompactCollector::StartCompaction calls back into Logger::allows_code_compaction() → same mutex, same thread → deadlock.

This is V8 bug 41497149, fixed 2024-05 by V8 commit a6eaf75741 ("[logging] Use RecursiveMutex for Logger", https://chromium-review.googlesource.com/c/v8/v8/+/5572952). The fix is in V8 ≥ 12.7, so Node 23/24 are safe, but every Node ≤ 22 is affected — including current latest v22.23.1 (checked: deps/v8/src/logging/code-events.h still uses plain base::MutexGuard), and Node 22 is in maintenance LTS until 2027.

Since the addon creates the CpuProfiler eagerly at module load, the exposure window is "every process start that imports the package" — larger apps (more compiled code to walk, more line-ends to allocate under the mutex) are more exposed.

Suggestions:

  1. Create the v8::CpuProfiler lazily on first profile start instead of at N-API Init — that both removes the cost for non-profiling processes and moves the risk out of the critical boot path (and applications that never enable profiling are never exposed).
  2. Failing that, a docs warning for Node ≤ 22 would help: the failure mode is brutal to diagnose (silent freeze before any app output, no crash, no event).

Happy to provide the full gdb/strace captures if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions