Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 4.3k
GH-17211: [C++] Add hash32 and hash64 scalar compute functions#45001
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kszucs
wants to merge
82
commits into
apache:mainChoose a base branch
from
kszucs:scalar-hash
base:main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Uh oh!
There was an error while loading. Please reload this page.
Open
Changes from all commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
80d840e
GH-17211: refresh history for scalar_hash kernel
drin 9260849
GH-17211: refresh history for updates to key_hash
drin e7252d0
GH-17211: refresh history for new compute fn infra
drin 5c3db19
GH-17211: refresh history of scalar hash benchmark
drin 49718f5
added KeyColumnArray alias
drin 37b19dd
task: revert style and other minor changes
drin 8f444b0
task: reducing support to some nested types
drin 5cba979
minor: added type check for ListArray
drin 312a3a5
Update the implementation to use column oriented hash-combine.
kszucs 949ffed
add hypothesis test case and fix list hashing
kszucs d5a1ba8
ensure to keep newly allocated arrays alive until hash gets computed
kszucs 79765d9
minor cleanup
kszucs d710fca
wrong include
kszucs 3b56994
add hash32 function
kszucs 618ec4a
add hash32 function and address review comments
kszucs 50e33ac
update testing suite
kszucs 1f907c2
generated struct tests
kszucs bf0b7a0
prepare adding hash32 tests
kszucs 866e4b0
add tests for hash32, dictionary encoded type, extension type
kszucs d9f0088
ensure that NULL and 0 hashes to different numbers
kszucs 691ccd3
fix the benchmarks
kszucs 2eb2ba4
update docs to highlight 0 return for NULL values
kszucs 2d5f93f
use resize instead of reserve
kszucs 5f9614a
fix R compute function filtering test
kszucs 9271efa
address review comments
kszucs fe2e206
update boolean test case
kszucs 661d82f
update exec checks
kszucs 40a710b
use Hasher::kHashBatchTempStackUsage
kszucs 3bb84a5
ensure that slices properly work
kszucs aeacc80
add tests for fixed_size_binary()
kszucs 0e14971
add rudimentary quality tests
kszucs c145283
test temporal types too
kszucs 28a9dca
test temporal and decimal types
kszucs cae89ef
add type matcher to explicitly raise NotImplemented for unsupported t…
kszucs 1293b40
update error message
kszucs 24a7dc6
update docs
kszucs 303d2ac
set a tolerance for the randomly generated numbers
kszucs ab78188
make msvc happy
kszucs b0d0b0a
Rename DCHECK macros to ARROW_DCHECK
kszucs 46dc7b8
Missing blank lines
kszucs f07c665
chore: move key_hash benchmarks to key_hash_benchmark.cc under compute
kszucs b457ec1
Apply suggestions from code review
kszucs 6528c69
use type singleton
kszucs 59a06ec
run clang format on scalar_hash_benchmark.cc
kszucs 53a62f9
run cmake format
kszucs 4ba82af
Address review comments and fix missing-declarations build failure
kszucs 39eecbe
Fix unreachable slicing check and remove unused hashing_rng in benchm…
kszucs 6489383
Fix nested-type null and hash-corruption bugs in hash32/hash64
kszucs 40711e8
Remove dead list-like branches from ToColumnArray
kszucs 25f0a0b
Fix independently-null nested fields hashing as non-zero garbage
kszucs 12fdde4
Size key_hash_benchmark's TempVectorStack from kHashBatchTempStackUsage
kszucs d3ff917
Hoist KeyColumnArray construction out of key_hash_benchmark's timed loop
kszucs c4e6a15
Fix GCC -Werror=range-loop-construct in scalar_hash_test.cc
kszucs 121d38d
Guard against a null values buffer in ToColumnArray's binary path
kszucs 80cb5ac
Add R test asserting hash32/hash64 are discoverable via list_compute_…
kszucs 39497f0
Avoid hashing unreferenced child data for sliced list/map arrays, fix…
kszucs c2b4915
Add hypothesis tests asserting hash32/hash64 respect Array.slice()
kszucs f1d03bb
Revert kAddend change to the shared HashIntImp; scope the null/0 fix …
kszucs b7a9791
Unwrap extension types in HashableMatcher so unsupported storage type…
kszucs ac63803
Add test coverage for zero/null collision across byte widths and FIXE…
kszucs 3c95c9d
Fix HashChild reusing a nested field's unshifted validity buffer
kszucs f9a65da
Avoid the bitmap copy in HashChild by omitting the validity buffer en…
kszucs 8d39471
Fix double blank lines around the new Hash Functions doc section
kszucs 85968a8
Avoid hashing unreferenced struct nested-field data, avoid gtest macr…
kszucs c1f30bb
Hoist the per-column vector out of KeyHashIntegers benchmark loops
kszucs eb9744d
Fix Hash32/Hash64 API docstrings
kszucs 8a736f3
Fix values-child offset cancellation bug in list/map hashing
kszucs 0f19f16
Fix valid struct rows coincidentally colliding with the null hash sen…
kszucs 5beac3f
Loosen RandomPrimitive's hash-quality tolerance for cross-platform RN…
kszucs c14e1ef
Fix independently-null fields of multi-field structs not hashing to 0
kszucs 1b550bd
Dedupe Hash64Struct{Small,Medium,Large}Strings benchmarks
kszucs 7a0e1b7
Make hash32/hash64 null-aware instead of reserving 0 for nulls
kszucs f309ce6
Fix zero-width fixed_size_binary hashing, and simplify
kszucs e35894e
Fix NullType struct fields not invalidating their row's hash
kszucs 59ca351
Add DoNotOptimize/ClobberMemory to key hash benchmarks
kszucs 5eedc72
Bulk-copy hash output validity, and fix doc nits
kszucs 1c09c5d
Reject unsupported nested child types during hash dispatch
kszucs abd254e
Drop unused builder_primitive.h include from hashing_benchmark
kszucs bd7b7ee
Route zero-width fixed_size_binary struct fields through HashArray
kszucs 33cdf3d
Add the new scalar-hash sources to the Meson build
kszucs 08ec6f2
Distinguish a null child element from one hashing to zero
kszucs 6c018be
Hash map keys even when the item is null
kszucs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.