Skip to content

Bump hyparquet-writer to 0.16.6 for the dictionary-encoding fix - #712

Merged
philcunliffe merged 1 commit into
masterfrom
bump-hyparquet-writer-0.16.6
Aug 11, 2026
Merged

Bump hyparquet-writer to 0.16.6 for the dictionary-encoding fix#712
philcunliffe merged 1 commit into
masterfrom
bump-hyparquet-writer-0.16.6

Conversation

@philcunliffe

Copy link
Copy Markdown
Contributor

Bumps the optional hyparquet-writer dep 0.16.5 -> 0.16.6, the release with the dictionary-encoding fix (hyparam/hyparquet-writer#35): the writer now keeps any dictionary that at least halves the encoded bytes instead of busting to PLAIN at the 1 MiB page cap. Server-side counterpart already merged (hyparam/hypaware-server#309); this covers the client's cache-tier writes.

The override: icebird 0.8.17 pins hyparquet-writer at exactly 0.16.5, and the cache write path goes through icebird, so a direct bump alone would leave a nested 0.16.5 where it matters. The override is scoped to icebird only; hypvector keeps its own 0.16.1 pin untouched. Drop the entry when icebird releases with its own bump.

Test updates: two format-parquet-clustering tests asserted the old writer's dictionary bust (it was the documented motivation for row-group clustering). They now assert the fixed behavior: the wide per-conversation column stays RLE_DICTIONARY with or without clustering, and clustering costs no material size (its remaining jobs are the peak-heap byte cap and read locality, per the updated rationale comment in the plugin). The old bust shape is kept as a regression canary.

Validation: typecheck green; 3,939/3,942 tests pass. The one failure (leave tears down a central layer whose active-slot pointer does not resolve (#623)) fails identically on unmodified master in two separate local checkouts, so it is pre-existing and unrelated (possibly macOS-specific).

🤖 Generated with Claude Code

…olumns
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@philcunliffe
philcunliffe merged commit a14ed1f into masterAug 11, 2026
9 checks passed
@philcunliffe
philcunliffe deleted the bump-hyparquet-writer-0.16.6 branch August 11, 2026 18:28
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@philcunliffe