Skip to content

b+tree: do not hand a fresh tree's whole node pool to its first commit - #166

Merged
psiha merged 1 commit into
bt/10-comparator-costfrom
fix/map-memory-clean-free-pool
Sep 11, 2026
Merged

b+tree: do not hand a fresh tree's whole node pool to its first commit#166
psiha merged 1 commit into
bt/10-comparator-costfrom
fix/map-memory-clean-free-pool

Conversation

@psiha

@psiha psiha commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Stacked on bt/10-comparator-cost (#165) — it only needs to be after it in the
stack, there is no logical dependency. Happy to rebase onto bt/9-bench-rigor or
master if that suits the ordering better.

The defect

map_memory( N ) threads the reserved pool onto the free list via
assign_nodes_to_free_pool, which calls free() on every node — and free() writes
the node header and marks it dirty.

At that moment the bit asserts something untrue. The tree has just been created, no
COW clone of it exists, and the storage already holds exactly what those nodes say.

commit_to() copies every dirty node, so the first commit of the tree's first
clone copied the entire reserved pool. And then it looks fine forever: commit_to()
clears the bit in its target, and later clones inherit the cleaned state. That
self-correction is precisely why a single-commit test cannot see this.

Measured

A counter in commit_to() tallying nodes / dirty / of which empty (num_vals == 0),
over map_memory( 20000 ) + 100 inserted values + two clone→mutate→commit cycles:

nodes dirty of which EMPTY
before, 1st commit 170 170 169
before, 2nd commit 170 1 0
after, 1st commit 170 2 1
after, 2nd commit 170 1 0

So the first commit was copying 169 empty free-pool nodes in order to move one node
of real change, and the fix makes the first commit cost what every later one costs.
The larger the reservation, the larger the one-off — a consumer that sizes the pool
from a known row count pays for the whole thing.

The fix

Clear the bits right after assign_nodes_to_free_pool( 0 ), in map_memory only.

Deliberately not in reserve() / reserve_additional(): those can run on a clone
that genuinely owes its newly added pool nodes to a target, so the same reasoning
does not hold there.

Testing

nodes_dirty() joins nodes_used() / nodes_reserved() so the property is
assertable from a test rather than only observable from a probe's stderr — and it is
a reasonable thing for a consumer to ask, being what a commit_to() of a clone will
cost.

bptree_cow.a_fresh_tree_owes_a_commit_nothing asserts it, and drives two commit
cycles so a regression cannot hide behind the self-correction above.

One thing the test deliberately does not assert: dirty <= used is not an
invariant. Taking a node off the free list also rewrites its successor's back-link
(new_nodeunlink_right), so a single allocation legitimately dirties two nodes.
I tried that assertion first and it failed at 2 vs 1; the comment now records why, so
the next person does not re-derive it.

Gate

Full suite, Release, Windows clang-cl: 2081 passed / 1 skipped / 0 failed (the
skip is the pre-existing bp_tree.replace_keys_inplace_stale_separator_underflow_repro).
2077 → 2081 is the four new assertions.

🤖 Generated with Claude Code

map_memory() threads the reserved pool onto the free list through
free(), and free() writes each node's header and marks it dirty.  At
that point the bit says something untrue: the tree has only just been
created, no COW clone of it exists, and the storage already holds what
those nodes say.

commit_to() copies every dirty node, so the first commit of the tree's
first clone copied the entire reserved pool.  It then looked fine
forever after, because commit_to() clears the bit in its target and
later clones inherit the cleaned state - which is exactly why a
single-commit test cannot see this.

Measured with a counter in commit_to() over map_memory(20000) plus 100
inserted values: the first commit copied 170 nodes, 169 of which held no
values at all, to move one node of real change.  With the bits cleared
it copies 2.

Cleared in map_memory() only.  reserve()/reserve_additional() can run on
a clone that genuinely owes its new pool nodes to a target, so the same
reasoning does not hold there.

nodes_dirty() joins nodes_used()/nodes_reserved() so the property is
assertable rather than only observable from a probe, and the test drives
two commit cycles so a regression cannot hide behind the self-correction
described above.  Note what the test does NOT assert: dirty <= used is
not an invariant, because taking a node off the free list also rewrites
its successor's back-link (new_node -> unlink_right), so a single
allocation legitimately dirties two nodes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@psiha
psiha force-pushed the fix/map-memory-clean-free-pool branch from aa358c4 to 9450aa0 Compare September 11, 2026 14:17
@psiha
psiha merged commit f044dad into bt/10-comparator-cost Sep 11, 2026
10 checks passed
@psiha
psiha deleted the fix/map-memory-clean-free-pool branch September 11, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant