Skip to content

b+tree: relieve an overflowing leaf into a sibling before splitting - #157

Open
psiha wants to merge 1 commit into
bt/4-entry-moverfrom
bt/5-local-redistribution
Open

b+tree: relieve an overflowing leaf into a sibling before splitting#157
psiha wants to merge 1 commit into
bt/4-entry-moverfrom
bt/5-local-redistribution

Conversation

@psiha

@psiha psiha commented Sep 9, 2026

Copy link
Copy Markdown
Owner

A split-on-full b+tree does not sit at its minimum fill — but it does not sit near full either. Measured by the accompanying characterisation test, 8M uint32 keys:

pattern before after leaf bytes/key
random, 512-byte nodes 69.5% 87.1% 5.94 → 4.74
sequential (either direction) 50.0% 99.2% 8.26 → 4.16
random, 4096-byte nodes 72.8% 89.9% 5.52 → 4.46
merge, interleaved (4096) 67.6% 88.2% 5.94 → 4.55
bulk / appended merge 100% / 99.9% unchanged

Random insertion settles at Yao's ln 2, not at 50%; sequential insertion settles at exactly 50%, which is where the waste actually is. The bulk paths are already at capacity and stay there.

So before splitting, an overflowing leaf now hands values to a same-parent sibling that still has room. This is Comer's local redistribution — and deliberately not a B*: no 2/3 floor, no 2-into-3 split, minimum fill untouched, no merge code changed. Structurally it is the exact dual of handle_underflow's borrow branches and uses the same idioms:

  • sibling existence resolved from parent_child_idx, never the level links — those cross parents;
  • the separator follows the values;
  • half the room found is taken, not all of it — a sibling emptied of slack merely moves the next split one node over, and a sibling left full would have to be split by the very insertion being relieved.

Leaves only, deliberately. A node's children carry a back-index into their parent, so relocating an inner node's children re-indexes and dirties every one of them — more than the split it would save.

Confirmed on the resident node pool, not just leaf bytes (hence nodes_used()/nodes_reserved()): pool bytes/key 6.09 → 4.87 random, 8.53 → 4.30 sequential.

Cost. Inserts are neutral-to-faster throughout. The one measured regression is find() in the 512-byte configuration, 1.08×, where intra-node search is a linear scan and a fuller leaf is a longer one; at 4096 bytes, where that search is binary, the same measurement is 0.95×. That is a property of occupancy, not of this policy — a bulk-built tree pays it too. The structural cost is that relieving leaves a node nearly full so it overflows again sooner; each event is individually cheaper, and to_move = room / 2 is the knob.

On by default; PSI_VM_BT_REDISTRIBUTE_ON_OVERFLOW=0 restores plain split-on-full.

Background — measured occupancy, the B*/Comer naming history, an eight-system prior-art table, and why this is not becoming a B* — is in doc/b+tree_occupancy_and_variants.md (#158).

Stacked on #156.

A split-on-full b+tree does not sit at its minimum fill, but it does not sit
near full either.  Measured by the accompanying characterisation test, 8M
uint32 keys: random one-by-one insertion settles at 69.5% leaf occupancy with
512-byte nodes and 72.8% with 4096-byte ones - Yao's ln 2 - and SEQUENTIAL
insertion, ascending or descending, settles at exactly 50%, which is where the
waste actually is.  The bulk paths are already at 100% and stay there.

So before splitting, an overflowing leaf now hands values to a same-parent
sibling that still has room.  This is the exact dual of handle_underflow's
borrow branches and uses the same idioms: sibling existence is resolved from
parent_child_idx, never the level links (those cross parents), and the
separator follows the values.  It hands over half the room found, not all of
it - a sibling emptied of slack just moves the next split one node over, and a
sibling left full would have to be split by the very insertion being relieved.

Leaves only, deliberately: a node's children carry a back-index into their
parent, so relocating an inner node's children re-indexes and dirties every
one of them, costing more than the split it would save.

    random, 512-byte nodes    69.5% -> 87.1%   5.94 -> 4.74 leaf bytes/key
    sequential                50.0% -> 99.2%   8.26 -> 4.16
    random, 4096-byte nodes   72.8% -> 89.9%   5.52 -> 4.46
    bulk / appended merge     unchanged at 100% / 99.9%

Confirmed on the resident node pool, not just leaf bytes (hence nodes_used()
and nodes_reserved()): 6.09 -> 4.87 pool bytes/key random, 8.53 -> 4.30
sequential.  The minimum fill is untouched and no merge code changes - this
raises occupancy without raising the floor, which is what would oblige a
3-into-2 merge.

Inserts are neutral-to-faster throughout.  The one measured cost is find() in
the 512-byte configuration, 1.08x, where intra-node search is a linear scan
and a fuller leaf is a longer one; at 4096 bytes, where that search is binary,
the same measurement is 0.95x.  That is a property of occupancy, not of this
policy - a bulk-built tree pays it too.

PSI_VM_BT_REDISTRIBUTE_ON_OVERFLOW=0 restores plain split-on-full.
@psiha
psiha force-pushed the bt/5-local-redistribution branch from 29ec4e0 to 262b915 Compare September 9, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant