Skip to content

repr: decode variable-length integers with arithmetic - #38502

Draft
petrosagg wants to merge 1 commit into
MaterializeInc:mainfrom
petrosagg:row-varint-arith
Draft

repr: decode variable-length integers with arithmetic#38502
petrosagg wants to merge 1 commit into
MaterializeInc:mainfrom
petrosagg:row-varint-arith

Conversation

@petrosagg

Copy link
Copy Markdown
Contributor

This PR changes the tags of each integer family to be consecutive and ordered by payload width, so the width can be derived by the tag's distance from the family's first tag. This allows us to completely remove Tag::actual_int_length, and read the payload with a branchless fixed eight byte load masked to that width. The load is guarded for the case where the datum is at the end of the Row buffer.

Additionally, for each integer family the largest variants (i.e NegativeIntX_X and NonNegativeIntX_X) are replaced by the corresponding previously unused IntX tag. This reduces the total number of tags from 94 to 84, leaving more free tags for dictionary encoding and shrinks the jump table by 7 entries. Reducing the SAFE_TAG_BASE constant is left for a separate PR though.

The PR includes the borrow_with benchmark from #38400, and tests that every integer boundary tags the width and sign the decoder reads back, and that the guarded tail path is taken at the end of a row and of a nested list.

Measured against upstream/main:

┌─────────────┬───────────┬───────────┬───────┐
│ borrow_with │ main │ this │ │
├─────────────┼───────────┼───────────┼───────┤
│ int4 │ 32.88 µs │ 14.54 µs │ 2.26× │
├─────────────┼───────────┼───────────┼───────┤
│ int8 │ 68.75 µs │ 34.60 µs │ 1.99× │
├─────────────┼───────────┼───────────┼───────┤
│ int32 │ 353.21 µs │ 133.10 µs │ 2.65× │
├─────────────┼───────────┼───────────┼───────┤
│ mixed3 │ 13.71 µs │ 8.38 µs │ 1.64× │
├─────────────┼───────────┼───────────┼───────┤
│ mixed12 │ 55.40 µs │ 32.93 µs │ 1.68× │
├─────────────┼───────────┼───────────┼───────┤
│ numeric8 │ 93.83 µs │ 57.03 µs │ 1.65× │
└─────────────┴───────────┴───────────┴───────┘
┌─────────────────────┬──────────┬──────────┬───────────┐
│ row │ main │ this │ │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_unpack_ints │ 27.50 ms │ 14.83 ms │ 1.85× │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_unpack_numeric │ 41.47 ms │ 30.88 ms │ 1.34× │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_iter_ints │ 2.50 ms │ 2.11 ms │ 1.19× │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_unpack_bytes │ 5.48 ms │ 5.49 ms │ unchanged │
├─────────────────────┼──────────┼──────────┼───────────┤
│ pack_pack_ints │ 579.2 µs │ 586.0 µs │ 1% slower │
└─────────────────────┴──────────┴──────────┴───────────┘

This PR changes the tags of each integer family to be consecutive and
ordered by payload width, so the width can be derived by the tag's
distance from the family's first tag. This allows us to completely
remove `Tag::actual_int_length`, and read the payload with a branchless
fixed eight byte load masked to that width. The load is guarded for the
case where the datum is at the end of the Row buffer.
Additionally, for each integer family the largest variants (i.e
`NegIntX_X` and `NonNegIntX_X`) are replaced by the corresponding
previously unused `IntX` tag. This reduces the total number of tags from
94 to 84, leaving more free tags for dictionary encoding and shrinks the
jump table by 7 entries. Reducing the `SAFE_TAG_BASE` constant is left
for a separate PR though.
The PR includes the `borrow_with` benchmark from MaterializeInc#38400, and tests that
every integer boundary tags the width and sign the decoder reads back,
and that the guarded tail path is taken at the end of a row and of a
nested list.
Measured against upstream/main:
┌─────────────┬───────────┬───────────┬───────┐
│ borrow_with │ main │ this │ │
├─────────────┼───────────┼───────────┼───────┤
│ int4 │ 32.88 µs │ 14.54 µs │ 2.26× │
├─────────────┼───────────┼───────────┼───────┤
│ int8 │ 68.75 µs │ 34.60 µs │ 1.99× │
├─────────────┼───────────┼───────────┼───────┤
│ int32 │ 353.21 µs │ 133.10 µs │ 2.65× │
├─────────────┼───────────┼───────────┼───────┤
│ mixed3 │ 13.71 µs │ 8.38 µs │ 1.64× │
├─────────────┼───────────┼───────────┼───────┤
│ mixed12 │ 55.40 µs │ 32.93 µs │ 1.68× │
├─────────────┼───────────┼───────────┼───────┤
│ numeric8 │ 93.83 µs │ 57.03 µs │ 1.65× │
└─────────────┴───────────┴───────────┴───────┘
┌─────────────────────┬──────────┬──────────┬───────────┐
│ row │ main │ this │ │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_unpack_ints │ 27.50 ms │ 14.83 ms │ 1.85× │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_unpack_numeric │ 41.47 ms │ 30.88 ms │ 1.34× │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_iter_ints │ 2.50 ms │ 2.11 ms │ 1.19× │
├─────────────────────┼──────────┼──────────┼───────────┤
│ sort_unpack_bytes │ 5.48 ms │ 5.49 ms │ unchanged │
├─────────────────────┼──────────┼──────────┼───────────┤
│ pack_pack_ints │ 579.2 µs │ 586.0 µs │ 1% slower │
└─────────────────────┴──────────┴──────────┴───────────┘
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@petrosagg

Copy link
Copy Markdown
ContributorAuthor

An initial test of stacking the prediction idea on top of this produces mostly wins and a regression in one of the benchmarks:

┌──────────┬───────────┬───────────┬──────────────────────┐
│ │ general │ predicted │ │
├──────────┼───────────┼───────────┼──────────────────────┤
│ int4 │ 14.33 µs │ 16.78 µs │ 17% slower │
├──────────┼───────────┼───────────┼──────────────────────┤
│ int8 │ 34.67 µs │ 28.29 µs │ 18% faster │
├──────────┼───────────┼───────────┼──────────────────────┤
│ int32 │ 132.57 µs │ 89.08 µs │ 33% faster │
├──────────┼───────────┼───────────┼──────────────────────┤
│ mixed3 │ 8.39 µs │ 8.47 µs │ unchanged (p = 0.15) │
├──────────┼───────────┼───────────┼──────────────────────┤
│ mixed12 │ 32.82 µs │ 27.91 µs │ 15% faster │
├──────────┼───────────┼───────────┼──────────────────────┤
│ numeric8 │ 52.95 µs │ 54.01 µs │ 2% slower │
└──────────┴───────────┴───────────┴──────────────────────┘

I'll have to take a closer look

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@petrosagg