Kafka Connect: Fix UUID conversion for Parquet writes - #17079
Conversation
RecordConverter.convertUUID() converted UUID values to byte[] when the target file format is Parquet. The Parquet UUID writer (ParquetValueWriters.uuids()) expects a java.util.UUID and converts to bytes internally, so writing a UUID column with the default file format threw ClassCastException: class [B cannot be cast to class java.util.UUID. The byte[] branch matched the writer contract before apache#11904 changed ParquetValueWriters' UUID writer to accept UUID directly; kafka-connect was not updated to follow. This removes the byte[] conversion so convertUUID always returns a UUID, matching ORC/Avro and the current Parquet writer contract. Generated-by: Claude Code
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
|
@thswlsqls same as #17080 — the stale bot closed this on 28 Aug without anyone reviewing it, and Reproduced on With your patch applied, the same path gives: Your diagnosis of why is right. The full One thing a reviewer should know, since it is not obvious from the thread: the alternative approach I would add that @bryanck could this be reopened? The change is @thswlsqls's. |
Closes #17076
Summary
RecordConverter.convertUUID()returnedbyte[]for UUID columns when the target file format is Parquet, but the Parquet UUID writer (ParquetValueWriters.uuids()) expects ajava.util.UUIDand converts to bytes itself, so writes threwClassCastException: class [B cannot be cast to class java.util.UUID.byte[]branch soconvertUUID()always returnsUUID, matching ORC (GenericOrcWriters.uuids()) and Avro, which already acceptUUIDdirectly.byte[]conversion matched the writer contract before PR Parquet: Add readers and writers for the internal object model #11904 changedParquetValueWriters' UUID writer to acceptUUIDdirectly;kafka-connectwas not updated to follow — this restores the correct contract.byte[]behavior, so it does not fix this bug; whichever of the two merges first, the other will need a rebase.Testing done
TestRecordConverter#testUUIDConversionWithParquetto assert the field equals the originalUUID, replacing theUUIDUtil.convert(UUID_VAL)byte[] expectation../gradlew :iceberg-kafka-connect:iceberg-kafka-connect:checkpasses —TestRecordConverter59/59, full module 122/122, 0 failures.AI Disclosure