b+tree: relieve an overflowing leaf into a sibling before splitting - #157
Open
psiha wants to merge 1 commit into
Open
b+tree: relieve an overflowing leaf into a sibling before splitting#157psiha wants to merge 1 commit into
psiha wants to merge 1 commit into
Conversation
This was referenced Sep 9, 2026
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 10:34
79ef721 to
e389feb
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 10:34
ffc76fe to
684c220
Compare
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 10:39
e389feb to
8790b31
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 10:39
684c220 to
58d1a41
Compare
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 10:43
8790b31 to
faa88ab
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 10:43
58d1a41 to
f1f4b93
Compare
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 10:49
faa88ab to
f807694
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 10:49
f1f4b93 to
2a37547
Compare
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 10:56
f807694 to
940536b
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 10:56
2a37547 to
b0eebe7
Compare
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 11:13
940536b to
b15e5f7
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 11:13
b0eebe7 to
29ec4e0
Compare
A split-on-full b+tree does not sit at its minimum fill, but it does not sit
near full either. Measured by the accompanying characterisation test, 8M
uint32 keys: random one-by-one insertion settles at 69.5% leaf occupancy with
512-byte nodes and 72.8% with 4096-byte ones - Yao's ln 2 - and SEQUENTIAL
insertion, ascending or descending, settles at exactly 50%, which is where the
waste actually is. The bulk paths are already at 100% and stay there.
So before splitting, an overflowing leaf now hands values to a same-parent
sibling that still has room. This is the exact dual of handle_underflow's
borrow branches and uses the same idioms: sibling existence is resolved from
parent_child_idx, never the level links (those cross parents), and the
separator follows the values. It hands over half the room found, not all of
it - a sibling emptied of slack just moves the next split one node over, and a
sibling left full would have to be split by the very insertion being relieved.
Leaves only, deliberately: a node's children carry a back-index into their
parent, so relocating an inner node's children re-indexes and dirties every
one of them, costing more than the split it would save.
random, 512-byte nodes 69.5% -> 87.1% 5.94 -> 4.74 leaf bytes/key
sequential 50.0% -> 99.2% 8.26 -> 4.16
random, 4096-byte nodes 72.8% -> 89.9% 5.52 -> 4.46
bulk / appended merge unchanged at 100% / 99.9%
Confirmed on the resident node pool, not just leaf bytes (hence nodes_used()
and nodes_reserved()): 6.09 -> 4.87 pool bytes/key random, 8.53 -> 4.30
sequential. The minimum fill is untouched and no merge code changes - this
raises occupancy without raising the floor, which is what would oblige a
3-into-2 merge.
Inserts are neutral-to-faster throughout. The one measured cost is find() in
the 512-byte configuration, 1.08x, where intra-node search is a linear scan
and a fuller leaf is a longer one; at 4096 bytes, where that search is binary,
the same measurement is 0.95x. That is a property of occupancy, not of this
policy - a bulk-built tree pays it too.
PSI_VM_BT_REDISTRIBUTE_ON_OVERFLOW=0 restores plain split-on-full.
psiha
force-pushed
the
bt/4-entry-mover
branch
from
September 9, 2026 12:01
b15e5f7 to
d0381cd
Compare
psiha
force-pushed
the
bt/5-local-redistribution
branch
from
September 9, 2026 12:01
29ec4e0 to
262b915
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A split-on-full b+tree does not sit at its minimum fill — but it does not sit near full either. Measured by the accompanying characterisation test, 8M
uint32keys:Random insertion settles at Yao's ln 2, not at 50%; sequential insertion settles at exactly 50%, which is where the waste actually is. The bulk paths are already at capacity and stay there.
So before splitting, an overflowing leaf now hands values to a same-parent sibling that still has room. This is Comer's local redistribution — and deliberately not a B*: no 2/3 floor, no 2-into-3 split, minimum fill untouched, no merge code changed. Structurally it is the exact dual of
handle_underflow's borrow branches and uses the same idioms:parent_child_idx, never the level links — those cross parents;Leaves only, deliberately. A node's children carry a back-index into their parent, so relocating an inner node's children re-indexes and dirties every one of them — more than the split it would save.
Confirmed on the resident node pool, not just leaf bytes (hence
nodes_used()/nodes_reserved()): pool bytes/key 6.09 → 4.87 random, 8.53 → 4.30 sequential.Cost. Inserts are neutral-to-faster throughout. The one measured regression is
find()in the 512-byte configuration, 1.08×, where intra-node search is a linear scan and a fuller leaf is a longer one; at 4096 bytes, where that search is binary, the same measurement is 0.95×. That is a property of occupancy, not of this policy — a bulk-built tree pays it too. The structural cost is that relieving leaves a node nearly full so it overflows again sooner; each event is individually cheaper, andto_move = room / 2is the knob.On by default;
PSI_VM_BT_REDISTRIBUTE_ON_OVERFLOW=0restores plain split-on-full.Background — measured occupancy, the B*/Comer naming history, an eight-system prior-art table, and why this is not becoming a B* — is in
doc/b+tree_occupancy_and_variants.md(#158).Stacked on #156.