2.0.0-rc.7 (performance): a projection derive that writes one root key of a large draft clones the whole raw on every derive, O(keys); plain stores use a prototype overlay

[thedanchez]

Summary

@solidjs/signals@2.0.0-rc.7, prod and dev builds. Performance, not correctness: the output is right, the cost is the surprise.

When a createProjection derive sets or deletes a root-level key of its draft, the engine clones the whole raw object before the write (ensurePB → cloneRaw, which walks every own property descriptor). It does this on the first root-level write of every derive, so a projection over a large keyed record pays O(keys) per derive no matter how small the change is. A nested write on the same projection is O(1), because the nested target is small. A plain createStore doing the identical root delete and set is O(1) after its first write: ensurePB gives plain stores a prototype overlay (Object.create(raw)) and only falls back to the clone for projection targets.

At 20 000 root keys that is about 17 ms per derive for touching a single key.

Reproduction (headless, only @solidjs/signals)

mkdir solid-projection-root-clone && cd solid-projection-root-clone
npm init -y >/dev/null && npm i solid-js@2.0.0-rc.7 >/dev/null
cat > repro.mjs <<'JS'
// A projection whose draft has many root keys: touching ONE root key per
// derive costs O(keys): the engine clones the whole raw on the first
// root-level write of a derive. A plain store deleting the same key is O(1).
// node rootclone.mjs [keys]
import { createProjection, createRoot, createSignal, createStore, flush } from "@solidjs/signals";
const KEYS = Number(process.argv[2] ?? 20000);
const ms = (fn) => { const t0 = performance.now(); fn(); return performance.now() - t0; };
const time = (label, run) => {
  const samples = [];
  for (let i = 0; i < 10; i++) samples.push(ms(() => { run(i); flush(); }));
  samples.sort((a, b) => a - b);
  console.log(`${label.padEnd(58)} median ${samples[5].toFixed(2).padStart(7)} ms   max ${samples[9].toFixed(2).padStart(7)} ms`);
};
const seed = () => Object.fromEntries(Array.from({ length: KEYS }, (_, i) => [`k${i}`, { n: i }]));
createRoot(() => {
  // 1. projection: one ROOT key deleted + re-added per derive
  const [tick, setTick] = createSignal(0, { ownedWrite: true });
  const rootProj = createProjection((draft) => {
    const i = tick();
    delete draft[`k${i}`];
    draft[`k${i}`] = { n: -i };
  }, seed(), { key: null });
  void rootProj.k0; flush();
  time(`projection: delete + set ONE root key (${KEYS} keys)`, (i) => setTick(i + 1));

  // 2. projection: one NESTED field written per derive (root untouched)
  const [tick2, setTick2] = createSignal(0, { ownedWrite: true });
  const nestedProj = createProjection((draft) => { const i = tick2(); draft[`k${i}`].n = -i; }, seed(), { key: null });
  void nestedProj.k0; flush();
  time(`projection: write ONE nested field (${KEYS} keys)`, (i) => setTick2(i + 1));

  // 3. plain store: the same root delete + set through the setter
  const [store, setStore] = createStore(seed());
  void store.k0; flush();
  time(`createStore: delete + set ONE root key (${KEYS} keys)`, (i) => setStore((draft) => { delete draft[`k${i}`]; draft[`k${i}`] = { n: -i }; }));
});
JS

node repro.mjs 20000
node --conditions=development repro.mjs 20000

Results (Node 24.13, macOS, Apple Silicon)

Prod build, 10 derives each, median and max:

projection: delete + set ONE root key (20000 keys)         median   17.57 ms   max   19.48 ms
projection: write ONE nested field (20000 keys)            median    0.01 ms   max    0.05 ms
createStore: delete + set ONE root key (20000 keys)        median    0.01 ms   max   15.05 ms

Dev build: 16.2 / 0.02 / 0.01 ms. At 2 000 keys the projection root write is 4.6 ms per derive.

The plain store's max is its first write (the one-time accessor scan and overlay setup); every later write is 0.01 ms. The projection pays the full cost on every derive.

Where it goes wrong (rc.7 source, store.ts)

function ensurePB(e) {
  let t = e.pb;
  ...
  if (t === null) {
    if (e.fam === null && !Array.isArray(e.v) && (e.sc ? !e.a : scanAccessorsOnce(e))) {
      t = e.pb = Object.create(e.v);   // plain store: prototype overlay, O(1)
      e.ovl = true;
    } else t = e.pb = cloneRaw(e.v, e); // projection target (fam !== null): full clone, O(keys)

cloneRaw calls Object.getOwnPropertyDescriptors on the raw and rebuilds the object from them. Projection targets always carry a fam, so they never take the overlay branch. The pb is discarded when the derive commits, so the next derive that touches the root clones again.

Nested writes stay cheap because ensurePB runs on the nested target, whose raw is small.

Impact

Solid Flow keeps its connection index as one projection: a record keyed by handle (nodeId, nodeId-type, nodeId-type-handleId) with one sub-record per key. At 10k nodes and 10k edges that is about 20k root keys. Reconnecting one edge moved its entries from one handle key to another, which deleted an emptied root key and added a new one, so every reconnect paid the clone: about 13 of the 34 ms per reconnect in the profile.

We worked around it by never mutating root keys on a reconnect: an emptied sub-record stays in place as {} and empties are pruned only when a never-seen key forces a root write anyway. That moved the reconnect to 21 ms. It is a shape constraint on the data structure rather than a fix, and it changed a documented detail of the record (a handle whose last connection is gone reads as {} instead of being absent).

Possible directions

• Give projection targets the same prototype overlay plain stores get when the raw has no accessors. Deletes would need to be recorded beside the overlay, the way the plain-store path already records a delete the overlay cannot shadow.
• Or keep the clone but make it incremental: copy the raw once and reuse the copy across derives, applying deletes and sets to it instead of rebuilding from descriptors each time.

Platform

• macOS 15 (Darwin 24.6.0), Apple Silicon
• Node 24.13.0
• solid-js@2.0.0-rc.7, @solidjs/signals@2.0.0-rc.7

[ryansolid]

Fixed in 12c3be9 (#3354) — thanks for the precise repro and the diagnosis, it was exactly right.

The #3044 prototype-overlay work that made plain stores O(written) had carved out families, so a projection derive's first root-level write still went through cloneRaw. ensurePB now admits non-optimistic families (projections, derived stores) to the overlay path; optimistic families, chained backings, arrays and accessor containers keep the clone. Your repro on the rebuilt prod build:

projection: delete + set ONE root key (20000 keys)   median  0.02 ms  (was 17.57)
projection: write ONE nested field (20000 keys)      median  0.02 ms
createStore: delete + set ONE root key (20000 keys)  median  0.00 ms

It also surfaced a real overlay bug on plain stores (a child's flatten could resurrect a slot the parent's earlier fold in the same batch had replaced or deleted), fixed in the same PR. A vitest bench in the #3352 shape and deterministic clone-count tests pin it. Lands in the next rc.

— Claude via Cursor