Skip to content

Keep the hand-rolled Psych parser; pin the yaml.v3-incompatible invariants - #28

Open
tannevaled wants to merge 1 commit into
mainfrom
refactor-parser-on-yaml-v3
Open

tannevaled wants to merge 1 commit into
mainfrom
refactor-parser-on-yaml-v3

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Verdict: KEEP (control-run proven)

This branch set out to base the Ruby object model + byte-exact MRI-Psych emit on
the reference gopkg.in/yaml.v3 parser (Node tree), shedding the hand-rolled
parser — the same idea as go-ruby-hcl2. As instructed, it was gated control-first.
The control run says do not swap, and this PR ships the KEEP outcome instead of
a refactor.

What was measured (yaml.v3 v3.0.1)

A prototype fed the reference parser's Node tree into this package's own scalar
resolution and !ruby/* tag mapping (never yaml.v3's Go-typed Decode, which
collapses Float/Integer and uses YAML-1.2 core-schema resolution), then replayed:

  1. The emitter corpus — every value Dump supports, parsed back through both
    loaders. Result: 100% parse-equivalent (30/30). yaml.v3 correctly reads
    everything this emitter writes.
  2. The full yaml/yaml-test-suite (402 accept/reject tests). Result: 115
    verdict changes — 61 regressions
    against the required Psych verdict vs 54 fixes.
    Net conformance loss (335 → 328) that also breaks the shrink-only ratchet.

Why it cannot move (bidirectional, first-hand)

Invariant This package yaml.v3 v3.0.1
Float 2.0 vs Integer 2 emit 2.0 / 2 Marshal(2.0)==Marshal(2)=="2" (collapses)
:name scalar Symbol("name") String(":name") (no Symbol)
!ruby/object: / !ruby/range *Object / *Range not reconstructible
%YAML 1.2 directive (27NA) accepts → "text" rejects ("incompatible document")
nil-keyed explicit mapping (2JQS) accepts rejects ("did not find expected key")
stray flow close [ a, b, c ] ] (4H7K) rejects accepts
junk after flow (KS4U) rejects accepts

The 61 regressions split 50 accept-axis (Psych YAML-1.1 leniencies yaml.v3 rejects)

  • 11 reject-axis (malformed blocks yaml.v3 accepts). The 53 reject-fixes and the 50
    accept-regressions are entangled in the same "trust yaml.v3's verdict" rule, so no
    fallback combinator (AND / OR / fallback-on-reject) is ratchet-safe. The reference
    is spec-compliant YAML 1.2; this package must be bug-for-bug Psych. This mirrors the
    sibling KEEP verdict for go-ruby-hcl2 (the Ruby value model does not fit go-cty).

What this PR contains

  • No refactor, no new dependency. Production code, the emitter (dump.go,
    byte-identical to main), and the 335/402 conformance are unchanged.
  • TestPsychModelInvariants (psychmodel_invariants_test.go) — a self-contained
    guard test pinning the five incompatible invariants above, so a future "wrap
    yaml.v3" attempt fails loudly. Verified to gate (fails when an invariant is
    violated).
  • README "Relationship to gopkg.in/yaml.v3" documenting the KEEP verdict.

Proof

  • Emit byte-exact preserved: dump.go/value.go/yaml.go diff vs main is empty;
    all TestDump* / TestOracle* golden tests green against ruby 4.0.5.
  • go test -race: 100.0% coverage; gofmt/go vet clean.
  • Conformance unchanged: 335/402 (83.33%), 0 regressions.

Not auto-merged (rbgo convention).

… invariants

A control run (gopkg.in/yaml.v3 v3.0.1) fed the reference parser's Node tree into
this package's own scalar resolution and !ruby/* tag mapping, then replayed the
emitter corpus and the full yaml/yaml-test-suite (402 tests). The verdict, matching
the sibling go-ruby-hcl2 finding, is KEEP: the reference is spec-compliant YAML 1.2,
Psych is bug-for-bug YAML 1.1, and the two cannot be reconciled without breaking the
load-bearing behaviour.

- Emitter: yaml.Marshal(2.0) == yaml.Marshal(2) == "2" — yaml.v3 cannot keep a Ruby
  Float distinct from an Integer, which rbgo's `require "yaml"` state/run-summary
  persistence depends on.
- Value model: yaml.v3 decodes `:name` to the String ":name" (no Symbol) and has no
  !ruby/object / !ruby/range reconstruction.
- Parser: swapping the block parser for yaml.v3 changed 115 accept/reject verdicts
  (61 regressions vs the required Psych verdict, 54 fixes) — a net conformance loss
  that also breaks the shrink-only ratchet, in both directions (yaml.v3 rejects a
  %YAML 1.2 directive and a nil-keyed mapping Psych accepts; accepts a stray flow
  close and post-flow junk Psych rejects).

The reference round-trips 100% of this emitter's output, but is not a drop-in
replacement for the Psych-faithful parser or emitter. No refactor and no new
dependency: only a self-contained guard test (TestPsychModelInvariants) that pins
the incompatible invariants, and a README "Relationship to gopkg.in/yaml.v3"
section. Production code, the emitter, and the 335/402 conformance are unchanged;
coverage stays 100%.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant