Skip to content

fix(compilers/openapi): treat an empty pointer segment as a token - #304

Merged
OmarAlJarrah merged 5 commits into
mainfrom
fix/openapi-empty-pointer-segment-hang
Aug 9, 2026
Merged

fix(compilers/openapi): treat an empty pointer segment as a token#304
OmarAlJarrah merged 5 commits into
mainfrom
fix/openapi-empty-pointer-segment-hang

Conversation

@OmarAlJarrah

@OmarAlJarrahOmarAlJarrah commented Aug 9, 2026

Copy link
Copy Markdown
Member

Summary

A path item whose $ref names its own location with a trailing separator —
#/components/pathItems/A/ — made Compile never return. No diagnostic, no error,
no progress: the process blocks forever inside the resolver. That is a denial of
service on untrusted spec input, since the document is legal YAML and legal OpenAPI
as far as anything upstream of the resolver is concerned. Under the CLI it surfaces
as Go's fatal error: all goroutines are asleep - deadlock! only because nothing
else is running; with any other goroutine alive the runtime detector cannot fire and
it is a plain hang.

The cycle pre-scan already has a guard for exactly this shape. chainReenters refuses
a reference whose pointer walks through a reference that is mid-resolve, because
speakeasy holds a reference's write lock across the pointer walk and read-locks every
reference the walk passes through, so re-entering one self-deadlocks on a
non-reentrant RWMutex. The guard is wired in and it is reached. It was being fed a
laundered input.

nodeview.PointerPath skipped every empty segment of a pointer:

forraw:=rangestrings.SplitSeq(pointer, "/") {
ifraw=="" {
continue
}

Only the leading empty string is an artifact of splitting on /. Every later one is
a real reference token. RFC 6901 §3 defines a pointer as *( "/" reference-token )
with reference-token = *( unescaped / escaped ) — the token may be empty, and it
names a member whose key is the empty string. speakeasy reads it the same way; its
parser splits strings.TrimPrefix(pointer, "/") and comments the case explicitly
("Empty parts are valid according to RFC 6901 - they represent empty string keys",
jsonpointer/navigation.go).

So #/components/pathItems/A/ does not stop at A; it descends throughA looking
for the key "" and never arrives. Skipping the token reported the walk as complete
and ending at A — and traverse excludes a complete walk's destination from the
re-entry test, correctly, because arriving at an on-chain node is an ordinary cycle
that speakeasy reports with a better message. The through-node was reclassified as a
destination, so the one node that had to be tested was the one node exempted.

The fix is in the tokenizer, not in the guard: split the way the resolver splits, and
let the empty token resolve or fail on its own terms. It fails here, which makes the
walk incomplete, which puts A back into the set of nodes the pointer passed through,
which yields chainReenters. The document is now refused with openapi/cyclic-ref
before the resolver ever runs.

Two smaller points that came with it:

  • A pointer with no tokens still names the root. Two spellings have none: the empty
    pointer, which is how a bare # arrives, and a lone /. The second is a deliberate
    departure from RFC 6901, which reads / as one empty token — getNavigationStack
    special-cases it to an empty navigation stack, and this walk models what the
    resolver walks rather than what the grammar admits. It is called out in the code.
  • The walk still splits lazily and still counts tokens against maxPointerSegments,
    so a pointer built to be long rather than to name anything is bounded exactly as
    before, without materializing its segments first.

Mechanism sweep

PointerPath is the compiler's only JSON-pointer walker, and scan.traverse is its
only production caller, so the deadlock's blast radius is closed. Every other site
that splits a pointer on / was checked:

  • schema.declaresResourceIDAbove skips empty segments the same way. Its consequence
    is different — a $id at a position whose pointer ends in an empty token is missed,
    so a $dynamicRef expansion crosses a resource boundary it should have degraded at
    — and so is its fix, since that walk reads $id before descending and must keep
    skipping the leading segment. Verified by probe and filed as openapi: $id resource-boundary walk skips empty pointer segments #302 rather than folded
    in here.
  • schema.branchPointerHint reads the last two segments without skipping anything;
    an empty index fails isDecimalIndex and the hint is declined. Correct as is.
  • overlay builds pointers by concatenation and looks them up exactly, never walking
    one. Not affected.
  • operation's split is over a URL path template, not a JSON pointer.

One verdict changes besides the hang

Reading the empty token also moves a schema-position ref of the same shape from
openapi/cyclic-ref to openapi/unresolved-ref. refCycles refuses a schema chain on
chainCycles alone; chainReenters is consulted only for the reference objects outside
a schema, since only those resolve through the Reference lock that deadlocks. Reading
/A/ as stopping at A made the shape a cycle, and reading it as descending through A
makes it what it is — a pointer naming a key that is not declared.

That is the more accurate verdict, and it is not a new way to hang: speakeasy resolves a
schema $ref as an oas3.JSONSchema rather than through that lock, and six schema
re-entrancy spellings were compiled to confirm the class reports rather than blocks.
TestCompile_SchemaEmptyPointerSegmentIsUnresolved pins it, and reverting the pointer fix
fails it naming openapi/cyclic-ref.

Test plan

  • Compile-level coverage comes from the reproducer table in
    compilers/openapi/cycles_test.go, which the fixture joins: no Go error, a nil
    document, an openapi/cyclic-ref error diagnostic. There is no per-test bound on
    it, deliberately. A regression here is a permanent deadlock rather than a failed
    assertion, and the bound this repository uses for that is already recorded and
    tuned — scripts/check-coverage.sh passes -timeout 90s explicitly, and it is the
    gate's only test invocation. A bespoke bound cannot improve on it while the same
    fixture is also compiled unbounded by the table and by internal/harness's corpus
    sweep: reverting the fix with one in place was measured, and both binaries still ran
    to the timeout.
  • TestPointerPath_EmptyTokenIsARealToken and
    TestPointerPath_EmptyTokenNamesTheEmptyKey pin both sides of the token rule: /a/
    does not resolve when no "" key is declared and reports a as passed through, and
    it resolves to the value when one is. TestPointerPath_LoneSeparatorNamesTheRoot
    pins the resolver-matching special case.
  • TestDetectCycles_EmptyPointerSegmentIsRefused encodes the reproducer at the scan
    level, next to the whitespace-normalization test that covers the sibling escape.
  • testdata/openapi/cycle_path_item_empty_segment.yaml joins the re-entrant-prefix
    family in both cycleReproducers tables and in the harness corpus's knownInvalid
    list, which also makes it a FuzzCycleDetector seed. Deleting the trailing / from
    the fixture reddens TestDetectCycles_Reproducers/path-item-empty-segment, so the
    fixture is testing what its name says.
  • Both declaration orders (path item first, component first) and both spellings
    (#/components/pathItems/A/ and #/paths/~1a/) were compiled through the CLI and
    refused with openapi/cyclic-ref; before the change the same inputs deadlocked.
  • Full gate green: gofmt, go vet, golangci-lint, go build, and the coverage
    gate at 100%.

Closes#238

TestCompile_EmptyPointerSegmentTerminates asserted exactly what
TestCompile_CyclicSpecDoesNotCrash/path-item-empty-segment already asserts
over the same fixture: no Go error, a nil document, and an openapi/cyclic-ref
error diagnostic. Its only additional content was a 10s bound, justified as
turning a regression from a suite that stops dead into a test that fails fast
and names the cause.
It does not do that. The same fixture is compiled without a bound by the
reproducer table beside it and by internal/harness's corpus sweep, so a
regression still runs both test binaries into `go test`'s timeout and prints a
stack dump. Reverting the pointer fix with the bounded test in place was
measured: it failed at 10.00s and the package still panicked at the timeout,
and internal/harness panicked on its own.
The bound the repository actually relies on for this failure mode is already
recorded and tuned — scripts/check-coverage.sh passes `-timeout 90s`
explicitly, and its comment says why. That script is the gate's only test
invocation, so 90s is always in force in CI.
Every other third-party citation in nodeview.go names the speakeasy version it
was read at, so a dependency bump has something to re-check against. The two
added with the empty-token rule did not.
tokenless also asserted that both tokenless spellings name the root but cited
the resolver only for the lone separator. The empty pointer takes a different
path there: references/resolution.go resolveAgainstDocument short-circuits it
to the root document before the pointer parser runs, which is worth naming
because getNavigationStack rejects an empty pointer outright.
Reading the empty token changes a second class of input, and it was neither
pinned nor stated. A schema $ref spelled '#/components/schemas/A/' used to be
refused openapi/cyclic-ref and is now reported openapi/unresolved-ref, because
the walk no longer stops at A: refCycles refuses a schema chain on chainCycles
alone, and the reclassified shape is chainReenters, which only the reference
objects outside a schema are checked for.
That asymmetry is correct — speakeasy resolves a schema $ref as an
oas3.JSONSchema rather than through the Reference lock that deadlocks — and the
new verdict is the more accurate one, since the pointer names a key that is not
declared. Six schema re-entrancy spellings were compiled to confirm the shape
reports rather than hangs. Pinning it makes the change deliberate: with the
pointer fix reverted the test fails naming openapi/cyclic-ref.
The comment stated why refCycles consults chainReenters only for the reference
objects outside a schema, as though the code recorded that rationale. It does
not. State the verified fact instead and name the revision it was read at:
openapi/reference.go's cacheMutex is the only lock held across a pointer walk
in v1.24.0, and jsonschema/oas3 carries no per-reference lock at all, which is
why the shape this hands the resolver reports rather than blocks.
@OmarAlJarrah
OmarAlJarrah merged commit 6434cd1 into mainAug 9, 2026
1 check passed
@OmarAlJarrah
OmarAlJarrah deleted the fix/openapi-empty-pointer-segment-hang branch August 9, 2026 07:40
OmarAlJarrah added a commit that referenced this pull request Aug 9, 2026
The conflict this branch predicted, resolved as it said: #304's
TestDetectCycles_EmptyPointerSegmentIsRefused calls Cycles(0, []byte(src)), and
Cycles now takes a sourceindex.Index, so the call becomes scanBytes(t, ...).
Two the branch could not predict. cycles_test.go: #304 and #310 added a test
where this branch adds a scanIndex helper, both at the same offset sharing a
closing brace — kept both. And #328 consolidated the openapi test scaffolding
into internal/openapitest after this branch was written, so its new
entry_internal_test.go calls sourceOf by the package-local name that no longer
exists; repointed, with the import added.
Byte-identical output re-proven against the merged tree rather than carried over
from the branch: 115 sources under testdata compiled through binaries built from
main and from this merge, capturing document, stderr and exit code each. diff -r
over the two trees is empty.
OmarAlJarrah added a commit that referenced this pull request Aug 11, 2026
The resource-boundary walk grew a pointer tokenizer of its own, which
duplicated nodeview.PointerPath — already fixed for this same empty-token
bug class in #304, in the layer below, a month earlier. Reusing it drops
the second tokenizer and with it three defects the copy carried: it read
a lone "/" as one empty token where the sibling walk reads it as the
root, it lost every boundary above a pointer with no leading "/", and
its loop had no explicit bound where PointerPath counts against
maxPointerSegments.
The walk also built a nodeview.View per call, discarding the expansion
memo the type exists to hold. The view now lives on AnchorIndex beside
the anchor memo, so one document builds one. Compiling merge-heavy specs
timed the same either way and the IR is byte-identical; what changes is
that the memo is reachable at all.
Sweeping the same empty-name mechanism turned up a second site the
original sweep missed. ids.ComponentEntry rejects an empty name, which
is deliberate — an entry keyed "" earns no named TypeID, as
testdata/conformance/openapi/empty-names.yaml records — but the
$dynamicAnchor path reported it as "declared at /components/schemas/
rather than on a component schema", which the document contradicts: that
pointer addresses a component schema. The verdict is unchanged; the
reason now says an empty name earns no named type to expand to.
Coverage: no committed spec combined an empty component name with $id
and $dynamicRef, so no oracle drove the fixed path. dynamic-ref.yaml now
writes that combination, and reverting the fix reddens the conformance
case. The empty-pointer unit case asserted nothing before — both
readings produced false — so the fixture now parks an $id under a root
member keyed "", which a walk that took "" for a token would descend
into and find.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openapi: a $ref naming its own location with a trailing empty segment hangs the compiler

1 participant

@OmarAlJarrah