hash_join.rs's create_hashes function panics with float columns with nightly rustc

[houqp]

Describe the bug

passing float arrays to create_hashes results in panics with a message "not implemented" when built with nightly rustc.

To Reproduce

Rustc:

✦ at 11:17:43 ❯ rustc --version
rustc 1.54.0-nightly (b663c0f4f 2021-05-29)

Code:

let random_state = RandomState::with_seeds(0, 0, 0, 0);
let schema = Schema::new(vec![Field::new("c1", DataType::Float32, false)]);
let batch = RecordBatch::try_new(
 Arc::new(schema),
 vec![Arc::new(Float32Array::from(vec![
 0.00005, 0.00002, 0.00003, 0.00001, 0.00004,
 ]))],
)
.unwrap();

let hashes_buff = &mut vec![0; batch.num_rows()];
let hashes = create_hashes(&[batch.columns()[0].clone()], &random_state, hashes_buff)?;

Expected behavior

It should not panic.

Additional context

Upstream issue: tkaitchuck/aHash#93

[Dandandan]

Interesting find @houqp . I think this needs to be fixed upstream, by e.g. casting to .
Any idea why it's using the fallback code of aHash at your end, what kind of cpu are you using?

[jorgecarleitao]

This is probably related to Rust's unability to support hash for f32 and f64, as Eq is still not stabilized for them (and Hash and Eq must be consistent).

[Dandandan]

This is probably related to Rust's unability to support hash for f32 and f64, as Eq is still not stabilized for them (and Hash and Eq must be consistent).

@houqp found an issue tkaitchuck/aHash#93 in upstream (trying to hash an usize from a len() call). I think it could be solved easily there by casting it to u64. It's not totally clear to me what's the difference between nightly and stable.

[houqp]

My CPU is Intel i7.

I think i found what's causing the difference between stable and nightly: tkaitchuck/aHash#93 (comment). This turned on the specialize feature in ahash, which triggered the fallback code path.

I think ahash just need to implement usize write in this case like @Dandandan suggested. There is nothing much we could do on our end without incurring extra overhead like converting the byte array to u32 or u64 before hashing it.