Skip to content

test(signals): add experimental semantic fuzzer - #3446

Draft
GabbeV wants to merge 1 commit into
solidjs:nextfrom
GabbeV:solid-2-fuzzer
Draft

GabbeV wants to merge 1 commit into
solidjs:nextfrom
GabbeV:solid-2-fuzzer

Conversation

@GabbeV

@GabbeV GabbeV commented Sep 14, 2026

Copy link
Copy Markdown

Summary

Adds the experimental semantic fuzzer used to investigate the recent Solid 2 consistency issues, so it can be run while developing fixes and extended as the semantics evolve. It generates reactive graphs and operation schedules, checks semantic invariants, and saves replayable, reduced findings for triage.

This draft is a handoff of the working experiment, not a proposal to make the current implementation a permanent repository fixture. The README starts with a short section for humans, followed by an extensive AI handoff covering the model and its limits, commands, artifacts, testing other branches, reduction, known cases, and how to revise rules without hiding unrelated failures. It also includes portable playground examples.

No Solid runtime changes are included. Everything is in one commit based on next at 63560a11.

How did you test this change?

  • pnpm --filter @solidjs/signals exec vitest run tests/semantics: 46 files, 327 tests passed.
  • pnpm --filter @solidjs/signals fuzz:types: passed.
  • Ran 1,000 generated cases: 966 passed, 22 failed semantic checks, and 12 produced policy findings; four reduced candidates, with no harness errors or limit results. Findings still require triage and are not counts of distinct Solid bugs.
  • Verified the documented sample, saved-case and indexed replays, persistent worker build, and reduction quality checks.
  • Fault calibration detected the injected drop-wake failure.

Validation was scoped to the fuzzer; the full repository test suite was not run for this handoff.

@changeset-bot

changeset-bot Bot commented Sep 14, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 026d5ac

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​fast-check@​4.9.010010010090100

View full report

@codspeed

codspeed Bot commented Sep 14, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 160 untouched benchmarks


Comparing GabbeV:solid-2-fuzzer (026d5ac) with next (63560a1)

Open in CodSpeed

ryansolid added a commit that referenced this pull request Sep 16, 2026
fix(signals): a render effect that stops reading a pending memo releases the memo's source (fuzzer #3446 P1)
github-actions Bot pushed a commit to yumemi-thomas/solid that referenced this pull request Sep 16, 2026
…er, the fuzzer's P1 as a cell and a pinned violation

Adds to the posture matrix:
- posture `gatedAway`: the reader is built behind a `show()` gate, then the
  gate closes — alive, no longer deriving from `x`. Records `x`,
  `isPending(x)` and the state's held SOURCE write after the gate closes,
  before any release.
- reader `effect`: a render effect reading `x` DIRECTLY (the reporter shape
  the fuzzer's reduction used; the matrix only had memo + effect-over-memo).
- state contract: `source` (the mainline signal whose write the flight holds)
  and `perturb` (a write made after the matrix reader exists, so the reader
  observes the flight it opens).
- a matrix-only state, "pending own async, observed only by the matrix
  reader (fuzzer P1)".

Result: fuzzer solidjs#3446 law P1 (21/1000 cases, one symptom group, case 854)
reproduces as exactly one cell — `effect × gatedAway` on that state: the
source's write stays held after the only reader gated away; with a memo
between the effect and the flight (`memo × gatedAway`) it releases. Pinned
`it.fails` in posture-born-held-and-observation.test.ts; recorded as spec O3
(violation, open): A15 / solidjs#3426 live reporter — "re-ran and no longer derives
from the source" is the fifth liveness case, to be fixed in one readerLive()
predicate rather than a fifth arm of reporterBlocksSource.

Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
github-actions Bot pushed a commit to yumemi-thomas/solid that referenced this pull request Sep 16, 2026
…ses the memo's source (fuzzer solidjs#3446 P1)

A pending REPORTER that recovers without its flight landing — its pass no
longer reads the source (a show() gate closed) — stops counting for the
transaction it reported to (A15 / solidjs#3426: the hold lasts while a live
reporter observes the flight). `reporterBlocksSource` already judged the
re-run effect dead; nothing RE-JUDGED the parked transaction, since the
flush that re-ran the effect had no active transaction, and the writes held
with it stayed staged for as long as the flight stayed up — forever, for one
that never lands.

recompute's tail now treats that recovery as the completion event it is and
wakes the transaction (wokenTransitions — the third site after disposal
solidjs#3372 and boundary reset solidjs#3375), skipped under the transaction's own flush,
which judges the landing itself. Reporters are stamped, so no unstamped
fallback is needed.

Found by the semantic fuzzer (solidjs#3446, law P1, 21/1000 cases in one symptom
group, case 854); reproduced as the posture matrix's `effect × gatedAway`
cell on the "observed only by the matrix reader" state — the memo-between
variant (`memo × gatedAway`) already released, which is what narrowed it to
the direct-effect reporter. Pin flipped from it.fails; spec O3 fixed.
+100 B minified (24,478 -> 24,578).

Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
ryansolid added a commit that referenced this pull request Sep 16, 2026
… retires the reporter and wakes every parked transaction (fuzzer #3446 P1, remaining forms)

Three corrections to how a parked transaction learns that a reporter
stopped observing the flight it waits on (A15 / #3426; spec O3):

1. `reporterBlocksSource`'s deps scan is bounded at `_depsTail` — this
   pass's reads. A staged pass keeps its previous deps linked past the tail
   until the commit trims them (A30), so a reporter whose pass had stopped
   reading the source in the same flush as the write still looked live
   through the kept dep, and the hold it kept was the commit that would have
   trimmed the dep that kept it. Rule 2's predicate reading Rule 3's
   deferral (fuzzer case 21).
2. recompute's tail retires a reporter when its pass DROPPED a dep, not
   only when it recovered from pending: a reporter registered by the
   stale-reader carve-out (heldFromStale, an initialized source refetching)
   displays the committed value and is never pending (fuzzer case 79).
3. The retirement wakes every parked transaction (wakeParked), not the
   reporter's stamp — the transaction waiting on it registered it without
   stamping it (a later write's hold over a flight an earlier step
   observed). One idle pass per parked transaction; done ones return.

Fuzzer, same campaign (seed 3289, 1000 cases): 994 pass / 0 fail / 6
policy, from 984 / 4 / 12. Same-flush pin flipped from it.fails.

Posture matrix: the runner records INVARIANT_VIOLATIONs per cell instead
of dying (new `invariant` column). With the parked-transaction leak gone,
quiescence checks run again and surface a pre-existing INV-4 — a
projection leaf's latest() shadow is stale on the flush right after its
root is disposed mid-refetch (spec O5; pinned it.fails as S3 in
posture-store-parity.test.ts; standalone repro on `next`). The two matrix
cells that trigger it are excluded until it is fixed; every other cell's
served value is unchanged.

Co-authored-by: Claude via Cursor <noreply@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
ryansolid added a commit that referenced this pull request Sep 16, 2026
fix(signals): reporter liveness reads this pass's deps; a dropped dep retires the reporter (fuzzer #3446 P1, remaining forms)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant