MPCoreDeveloper · MPCoreDeveloper · Sep 2, 2026 · Aug 30, 2026 · Aug 30, 2026 · Aug 30, 2026
diff --git a/.gitignore b/.gitignore
@@ -57,6 +57,7 @@ dlldata.c
 
 # Benchmark Results
 BenchmarkDotNet.Artifacts/
+# Comparative benchmark harness output (tests/benchmarks/SharpCoreDB.Benchmarks.Comparative)
 results/
 
 # .NET Core

diff --git a/ROADMAP.md b/ROADMAP.md
@@ -131,9 +131,12 @@
 >> **Why:** v2.0 closed the read/insert gap; single-row UPDATE/DELETE is still ~5–7x behind SQLite
 >> because row-store writes are append-on-update instead of in-place.
 
-- In-place record updates (avoid append-on-update) for row stores
-- Fixed-width record layout for hot tables (SQLite-style C record format)
-- Eliminate read-modify-write in `UpdateMultiple`
+- ✅ **In-place record updates for columnar/append-only (#6)** — fixed-width/unchanged-length
+ records overwrite their slot; no file growth
+- ✅ **Single-pass SQL DELETE/UPDATE (#7/#8)** — no more double materialization for RETURNING /
+ `CHANGES()`; PK fast path in `Delete`/`DeleteMultiple`/`UpdateMultiple`
+- ⬜ Fixed-width record layout for hot tables (SQLite-style C record format)
+- ⬜ Storage-level DELETE reuse (free-slot reuse / compaction on PageBased)
 - Track in [`docs/performance/V2_PERFORMANCE_PLAN.md`](docs/performance/V2_PERFORMANCE_PLAN.md)
 
 ---

diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
diff --git a/docs/PROJECT_STATUS.md b/docs/PROJECT_STATUS.md
@@ -55,8 +55,18 @@ SharpCoreDB core .NET packages are release-labeled on `2.0.0` and build successf
 
 ## Roadmap / TODO (v2.1)
 
-- [ ] **Close UPDATE/DELETE gap vs SQLite** — in-place record updates, fixed-width record layout
- for hot tables, eliminate read-modify-write in `UpdateMultiple`.
+- [ ] **Close UPDATE/DELETE gap vs SQLite** (in progress — details in
+ `docs/performance/V2_PERFORMANCE_PLAN.md` §3.4 / §3.5):
+ - ✅ **In-place UPDATE for columnar/append-only (Issue #6)** — fixed-width / unchanged-length
+ records overwrite their existing slot (`TryUpdateInPlace`); no new version, no file growth.
+ - ✅ **Single-pass SQL DELETE/UPDATE (Issue #7/#8)** — `DeleteAffectedRows` / `UpdateAffectedCount`
+ return the affected rows/count from the table operation itself, so the SQL paths no longer
+ materialize matching rows twice for RETURNING / change-tracking.
+ - ✅ **PK fast path in `Delete` / `DeleteMultiple` / `UpdateMultiple`** — a simple `pk = value`
+ WHERE resolves via the primary-key B-tree directly (single search + one read) instead of
+ full-row materialization + per-row re-search.
+ - [ ] Fixed-width record layout for hot tables (SQLite-style C record format)
+ - [ ] Storage-level DELETE reuse (free-slot reuse / compaction on PageBased deletes)
 - [ ] **.NET 11 / C# 15 migration** (after Nov 2026 GA) — Runtime Async, AVX-VNNI-512/SVE2 behind
  `SIMD_ENABLED`, optional Zstandard compression.
 - [ ] **Native AOT warning cleanup** — interface-based B-tree factory (replace `GetMethod`/

diff --git a/docs/backlog/NET11_C15_BACKLOG.md b/docs/backlog/NET11_C15_BACKLOG.md
@@ -0,0 +1,40 @@
+# V2 Backlog — .NET 11 / C# 15-gebonden werk
+
+**Status:** Gevuld · 2026-08-31 · v2.1-lijn (`release/v2.1.0.0`)
+**Doel:** Items hieronder kunnen pas worden uitgevoerd zodra de bijbehorende runtime-/compiler-
+functies beschikbaar zijn (target: .NET 11 GA, november 2026). Alles wat **nu al kan** en de
+performance verhoogt om het SQLite-gat te dichten, wordt buiten deze backlog uitgevoerd — zie
+[`docs/performance/V2_PERFORMANCE_PLAN.md`](../performance/V2_PERFORMANCE_PLAN.md).
+
+## Waarom deze items geblokkeerd zijn
+
+Zie `V2_PERFORMANCE_PLAN.md` §4.0 (preview-7-metingen, 2026-08-30):
+
+| Functie | In preview 7? | Blokkade |
+|---|---|---|
+| Numeriek `LangVersion 15.0` | ❌ | Preview-compiler geeft `CS1617`; `LangVersion latest` is de tijdelijke workaround |
+| Runtime Async | ✅ (net11) | automatic; geen code-wijziging tot GA-baseline |
+| AVX-VNNI-512 / Arm SVE2 | ⚠️ | SVE2 is `SYSLIB5003` evaluation-only; SVE2 uitstellen tot GA |
+| SIMD lane APIs | ✅ (preview 7) | vereist een columnar-layout refactor, geen point-edit |
+| Zstandard (`ZstdCompressor`) | ❌ | niet in preview 7; uitgesteld tot later preview/GA |
+| IEEE 754 `Decimal32/64/128` | ❌ | niet in preview 7; uitgesteld tot GA |
+| C# 15 union types / closed hierarchies | ⚠️ | nog niet gestabiliseerd; valideren vóór AST-refactor |
+
+## Backlog
+
+| # | Item | Afhankelijkheid | Aanraakgebied (indicatie) |
+|---|------|-----------------|---------------------------|
+| B1 | `LangVersion latest` → `15.0` | .NET 11 GA | `Directory.Build.props`, `global.json` |
+| B2 | Runtime-native async in async hot paths | Runtime Async (net11 GA) | `Execute*Async`, `InsertBatchAsync`, `ExecuteBatchSQLAsync`, server-paden |
+| B3 | AVX-VNNI-512 (x64) + Arm SVE2 intrinsics achter `SIMD_ENABLED`-guards | AVX-VNNI-512 net11; SVE2 eval-only tot GA | `DistanceMetrics`, `SimdHelper`, vector search (HNSW) |
+| B4 | SIMD lane APIs (`Zip`/`Unzip`/`CreateGeometricSequence`/`Concat`) in columnar codecs | APIs ✓ in preview 7, maar vereist columnar-layout refactor | Delta, Gorilla, XorFloat, RLE, bit-packing, SIMD row scanning |
+| B5 | Zstandard WAL/page-compressie (opt-in, default uit) | `ZstdCompressor` in `System.IO.Compression` | WAL + page-compressie (net als bestaande Brotli/GZip block-compressie) |
+| B6 | IEEE 754 `Decimal32/64/128` + `INumberBase<TSelf>.TryParsePartial` | runtime (niet in preview 7) | decimal-column parsing/serialisatie |
+| B7 | C# 15 union types / closed hierarchies voor de SQL-AST | compiler-stabilisatie (Phase 4) | `SqlParser`, planner, AST/SQL-node design |
+| B8 | Automatic JIT / NativeAOT-dispatch wins meten | net11 GA (geen code-wijziging) | re-benchmark + `V2_PERFORMANCE_PLAN.md` §3.2 bijwerken |
+
+## Niet performance-gerelateerd (apart bijhouden, niet deze backlog)
+
+- Native AOT-waarschuwings-cleanup: B-tree factory ipv. reflectie, `ParseVectorValue`/`.scdb` JSON
+ naar source-generated context.
+- `SingleFileDatabase` → `IMetadataProvider` pariteit (metadata-detectie via `db is IMetadataProvider`).
diff --git a/docs/benchmarks/FIXED_WIDTH_BENCHMARK.md b/docs/benchmarks/FIXED_WIDTH_BENCHMARK.md
@@ -0,0 +1,72 @@
+# Fixed-Width vs Legacy — benchmark results
+
+Run: 2026-09-01, .NET 11.0.0-preview.7, Windows, Release.
+Command: `dotnet run --project tests/benchmarks/SharpCoreDB.Benchmarks.Comparative -- --fixedwidth`
+
+The same workloads run against a legacy (variable-length records) database and a fixed-width
+database (directory-mode Columnar, `DatabaseConfig.FixedWidthRecordLayout = true`). Both databases
+use identical settings (no encryption, memory mapping, page cache).
+
+## Results
+
+| Workload | Metric | Legacy | Fixed-width | Win |
+|---|---|---|---|---|
+| A · 10,000 growing variable-column updates | elapsed | 17.99 s | **2.12 s** | **~8.5× faster** |
+| A · 10,000 growing variable-column updates | storage growth (post-auto-compact) | 0.0 KB | 20.4 KB | ≈ |
+| B · 1,000 variable updates + arena compaction | elapsed | 0.59 s | **0.14 s** | **~4× faster** |
+| B · 1,000 variable updates + arena compaction | storage growth | 23 B | 17 B | ≈ |
+| C · 30 full scans, non-indexed `WHERE category = -1` over 100,000 rows | time per query | 6.31 ms | **2.63 ms** | **~2.4× faster** |
+| D · 100,000 batch inserts | throughput | 242,487 rows/s | 208,692 rows/s | ~14% slower |
+
+## Interpretation
+
+- **Updates (growing variable values) — ~8.5× faster.** Legacy appends a new record per growing
+ update and pays for full `.dat` compactions (1000-update threshold); fixed-width keeps the `.dat`
+ constant (in-place overwrite), grows only the overflow arena, and compacts only the arena (B1/B3).
+- **Non-indexed full-scan WHERE — ~2.4× faster.** Fixed-width reads the predicate column at its
+ constant slot offset (numeric early-WHERE) or compares the arena payload (string early-WHERE) and
+ skips full-row deserialization for non-matches (B4).
+- **Variable updates + compaction — ~4× faster.** The arena copy-on-compact is cheaper than a
+ `.dat` rewrite (B3).
+- **Inserts — ~14% slower.** Fixed-width writes each variable value into the overflow arena
+ (payload encoding + free-list bookkeeping); for insert-heavy workloads the legacy format is
+ slightly faster. This is the expected trade-off: fixed-width targets update-heavy / point-read
+ workloads.
+
+## Comparative CRUD vs SQLite/LiteDB (post B7 update-path work)
+
+AppendOnly engine (`--engine=appendonly`), 100K inserts / 10K reads / 10K updates / 10K deletes:
+
+| Database | INSERT ops/s | READ ops/s | UPDATE ops/s | DELETE ops/s |
+|---|---|---|---|---|
+| SharpCoreDB (SQL) | 91,873 | 65,858 | 37,203 | 60,211 |
+| SharpCoreDB (Direct) | 116,146 | 141,293 | 46,189 | 61,554 |
+| SharpCoreDB (StructRow) | 135,322 | 120,224 | – | – |
+| SQLite | 148,654 | 95,143 | 281,072 | 351,863 |
+| LiteDB | 78,569 | 13,721 | 9,641 | 14,710 |
+
+PageBased engine (`--engine=pagebased`):
+
+| Database | INSERT ops/s | READ ops/s | UPDATE ops/s | DELETE ops/s |
+|---|---|---|---|---|
+| SharpCoreDB (SQL) | 123,995 | 30,807 | 69,574 | 124,176 |
+| SharpCoreDB (Direct) | 124,917 | 39,669 | 102,160 | 175,887 |
+| SharpCoreDB (StructRow) | 209,068 | 51,438 | – | – |
+| SQLite | 146,301 | 94,523 | 266,673 | 372,029 |
+| LiteDB | 71,677 | 13,616 | 10,337 | 14,061 |
+
+- **vs LiteDB: SharpCoreDB wins every workload** (1.5–8×).
+- **vs SQLite:** SharpCoreDB wins on INSERT (StructRow) and on READ (AppendOnly Direct + SQL after
+ B9); SQLite remains ~6–8× faster on UPDATE/DELETE. The batch UPDATE path is now in-place and
+ **deserialize-free on the hot path** (only the changed fields are patched at their slot
+ offsets) — the remaining gap is the per-statement parser + write-behind bookkeeping vs SQLite's
+ specialized b-tree writes.
+
+### Point-read micro-benchmark (`--readtest`, median of 7 × 10K reads on 100K rows)
+
+| Path | ops/s | notes |
+|---|---|---|
+| SQL `SELECT * FROM docs WHERE name = @name` | ~120,000–166,000 | B9 direct hash-index lookup (was ~65,000) |
+| Direct `FindByIndex("docs", "name", …)` | ~160,000–188,000 | reference |
+| SQL/Direct overhead | 1.1–1.5× | was ~2× |
+| SQLite (same workload) | ~95,000 | — |
diff --git a/docs/benchmarks/V198_V20_V21_PERFORMANCE_COMPARISON.md b/docs/benchmarks/V198_V20_V21_PERFORMANCE_COMPARISON.md
@@ -61,6 +61,7 @@
 - SQL UPDATE is ~5–10× slower than SQLite in every version (2.x: 27–43K vs SQLite 218–280K).
 - SQL DELETE is ~5–17× slower (2.x: 21–61K vs SQLite 295–364K), high variance.
 - Root cause is structural: SharpCoreDB's row-store updates/deletes are row-copy based, while SQLite uses fixed-length C records with direct field offsets and in-place writes. This is the targeted v2.1+ engine work (in-place records), **not** something the runtime or allocations fix.
+- **Progress (#6, 2026-08-31):** the in-place UPDATE engine landed on `release/v2.1.0.0` (`3d4cee77` + `68cb5dab`) and `release/v2.0.0.0` (`116fc30e` + `8a13ba2b`). On the columnar/append-only engine, **fixed-width UPDATEs no longer append a new version**: measured ~3.5–5.3K ops/s with **0 file growth** vs ~1.5K ops/s with +90 KB growth per 2,000 updates before. Variable-width updates fall back to the append path unchanged. See `V2_PERFORMANCE_PLAN.md` §3.4. The DELETE gap and the numbers above (measured on the default single-file path) are unchanged by this work.
 
 ### 3.4 Versus competitors
 - SharpCoreDB 2.x beats **LiteDB on every operation** (~5–8× reads, ~4–5× updates, ~3–9× deletes).

diff --git a/docs/performance/V2_PERFORMANCE_PLAN.md b/docs/performance/V2_PERFORMANCE_PLAN.md
@@ -192,7 +192,31 @@ What changed:
 > (2–26× over scalar). See
 > [`docs/benchmarks/AVX512_2026-09-01.md`](../benchmarks/AVX512_2026-09-01.md).
 
-
+### 3.5 #7/#8 single-pass DML — SQL DELETE/UPDATE no longer materialize twice (2026-08-31, `release/v2.1.0.0`)
+
+The SQL DELETE path previously materialized every matching row **twice** per statement:
+`ExecuteDelete` ran a full `Select` (for RETURNING + affected-count) and then `Table.Delete`
+re-scanned/re-deserialized the same rows. The SQL UPDATE path was worse: a full `Select().Count`
+for change-tracking, the update pass itself, and — for RETURNING — a second full `Select`.
+
+Changes:
+
+- **`ITable.DeleteAffectedRows(where)`** — default implementation keeps the historic two-pass
+ behavior for third-party `ITable` implementers; `Table` and `SingleFileTable` override with a
+ single pass (delete AND return the affected pre-delete rows). `ExecuteDelete` now uses it:
+ one scan, RETURNING + count from the same rows.
+- **`ITable.UpdateAffectedCount(where, updates)`** — same default/override pattern; applies the
+ update and returns the affected count. `ExecuteUpdate` now uses it; the separate `Select().Count`
+ pass is gone (RETURNING still re-selects, only when requested).
+- **PK fast path (Issue #7) extended to `DeleteMultiple` and `UpdateMultiple`** — a simple
+ `pk = value` WHERE on a columnar table resolves via the PK B-tree directly (single search + one
+ read) instead of `SelectInternal` full-row materialization + a per-row PK re-search. Range /
+ compound / non-indexed WHERE clauses bypass the fast path and keep their (correct) generic
+ behavior — `TryParseSimpleWhereClause` only accepts a plain `col = value`.
+
+Regression coverage: `DmlSinglePassTests` (affected counts, RETURNING pre-delete rows, range +
+non-indexed fallbacks, batch PK deletes/updates) + the existing RETURNING / `CHANGES()` tests.
+Full suite green: **1,644 tests, 0 failures** (16 skipped).
 
 ---
 

diff --git a/src/SharpCoreDB/DataStructures/FixedWidthCodec.cs b/src/SharpCoreDB/DataStructures/FixedWidthCodec.cs
@@ -0,0 +1,154 @@
+// <copyright file="FixedWidthCodec.cs" company="MPCoreDeveloper">
+// Copyright (c) 2026 MPCoreDeveloper and GitHub Copilot. All rights reserved.
+// Licensed under the MIT License. See LICENSE file in the project root for full license information.
+// </copyright>
+namespace SharpCoreDB.DataStructures;
+
+using System.Buffers.Binary;
+using System.Collections.Generic;
+
+/// <summary>
+/// Shared fixed-width record codec (out-of-line overflow model). Every column occupies a constant
+/// slot in the record's fixed part: fixed-size columns store <c>[null-flag(1)][payload]</c> inline,
+/// variable-length columns (String / Blob) store a 5-byte slot <c>[null-flag(1)][arena-offset(4)]</c>
+/// referencing a block in the overflow arena. Used by both the directory-mode <see cref="Table"/>
+/// and the single-file (<c>.scdb</c>) table so the two storage modes share one record format.
+/// </summary>
+public static class FixedWidthCodec
+{
+ /// <summary>Serializes a row dictionary into a fixed-width record (variable values → arena).</summary>
+ public static byte[] SerializeRow(
+ Dictionary<string, object> row,
+ IReadOnlyList<string> columns,
+ IReadOnlyList<DataType> types,
+ FixedWidthRecordLayout layout,
+ IOverflowArena arena)
+ {
+ var buffer = new byte[layout.FixedSize];
+ var span = buffer.AsSpan();
+
+ for (int i = 0; i < columns.Count; i++)
+ {
+ var slot = span.Slice(layout.Offsets[i], layout.SlotSizes[i]);
+ var value = row.TryGetValue(columns[i], out var v) ? v : DBNull.Value;
+
+ if (layout.IsVariable[i])
+ {
+ if (value == null || value == DBNull.Value)
+ {
+ slot[0] = 0;
+ BinaryPrimitives.WriteInt32LittleEndian(slot[1..], 0);
+ }
+ else
+ {
+ var payload = Table.EncodeVariablePayload(types[i], value);
+ var offset = arena.Write(payload);
+ slot[0] = 1;
+ BinaryPrimitives.WriteInt32LittleEndian(slot[1..], (int)offset);
+ }
+ }
+ else
+ {
+ _ = Table.WriteTypedValueToSpan(slot, value, types[i]);
+ }
+ }
+
+ return buffer;
+ }
+
+ /// <summary>Deserializes a fixed-width record into a row dictionary (variable values ← arena).</summary>
+ public static Dictionary<string, object> DeserializeRow(
+ ReadOnlySpan<byte> data,
+ IReadOnlyList<string> columns,
+ IReadOnlyList<DataType> types,
+ FixedWidthRecordLayout layout,
+ IOverflowArena arena)
+ {
+ var row = new Dictionary<string, object>(columns.Count, System.StringComparer.Ordinal);
+
+ for (int i = 0; i < columns.Count; i++)
+ {
+ if (layout.Offsets[i] + layout.SlotSizes[i] > data.Length)
+ {
+ break; // truncated / corrupt record
+ }
+
+ var slot = data.Slice(layout.Offsets[i], layout.SlotSizes[i]);
+ if (layout.IsVariable[i])
+ {
+ if (slot[0] == 0)
+ {
+ row[columns[i]] = DBNull.Value;
+ }
+ else
+ {
+ var offset = BinaryPrimitives.ReadInt32LittleEndian(slot[1..]);
+ var payload = arena.Read(offset);
+ row[columns[i]] = payload is null ? DBNull.Value : Table.DecodeVariablePayload(types[i], payload);
+ }
+ }
+ else
+ {
+ row[columns[i]] = Table.ReadTypedValueFromSpan(slot, types[i], out _);
+ }
+ }
+
+ return row;
+ }
+
+ /// <summary>Collects the arena offsets referenced by a fixed-width record's variable slots.</summary>
+ public static void CollectVariableOffsets(byte[] record, FixedWidthRecordLayout layout, HashSet<long> live)
+ {
+ for (int i = 0; i < layout.ColumnCount; i++)
+ {
+ if (!layout.IsVariable[i])
+ {
+ continue;
+ }
+
+ var slot = layout.Offsets[i];
+ if (slot + 5 > record.Length || record[slot] == 0)
+ {
+ continue; // truncated or null slot
+ }
+
+ var blockOffset = BinaryPrimitives.ReadInt32LittleEndian(record.AsSpan(slot + 1, 4));
+ // NOTE: offset 0 is a valid block offset (first arena block) — the flag byte above
+ // already excluded NULL slots, so collect every referenced offset unconditionally.
+ live.Add(blockOffset);
+ }
+ }
+
+ /// <summary>
+ /// Returns a copy of a fixed-width record with its variable slots re-pointed through the
+ /// compaction mapping, or null when no slot moved.
+ /// </summary>
+ public static byte[]? RepointVariableSlots(byte[] record, FixedWidthRecordLayout layout, Dictionary<long, long> mapping)
+ {
+ byte[]? result = null;
+
+ for (int i = 0; i < layout.ColumnCount; i++)
+ {
+ if (!layout.IsVariable[i])
+ {
+ continue;
+ }
+
+ var slot = layout.Offsets[i];
+ if (slot + 5 > record.Length || record[slot] == 0)
+ {
+ continue;
+ }
+
+ var blockOffset = BinaryPrimitives.ReadInt32LittleEndian(record.AsSpan(slot + 1, 4));
+ // NOTE: offset 0 is a valid block offset (first arena block) — re-point it like any other.
+ if (mapping.TryGetValue(blockOffset, out var newOffset) && newOffset != blockOffset)
+ {
+ result ??= (byte[])record.Clone();
+ BinaryPrimitives.WriteInt32LittleEndian(result.AsSpan(slot + 1, 4), (int)newOffset);
+ }
+ }
+
+ return result;
+ }
+}