Skip to content

Kafka Connect: Fix UUID conversion for Parquet writes - #17079

Closed
thswlsqls wants to merge 1 commit into
apache:mainfrom
thswlsqls:fix/kafka-connect-uuid-parquet-conversion
Closed

Kafka Connect: Fix UUID conversion for Parquet writes#17079
thswlsqls wants to merge 1 commit into
apache:mainfrom
thswlsqls:fix/kafka-connect-uuid-parquet-conversion

Conversation

@thswlsqls

@thswlsqls thswlsqls commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Closes #17076

Summary

  • RecordConverter.convertUUID() returned byte[] for UUID columns when the target file format is Parquet, but the Parquet UUID writer (ParquetValueWriters.uuids()) expects a java.util.UUID and converts to bytes itself, so writes threw ClassCastException: class [B cannot be cast to class java.util.UUID.
  • Removes the byte[] branch so convertUUID() always returns UUID, matching ORC (GenericOrcWriters.uuids()) and Avro, which already accept UUID directly.
  • The byte[] conversion matched the writer contract before PR Parquet: Add readers and writers for the internal object model #11904 changed ParquetValueWriters' UUID writer to accept UUID directly; kafka-connect was not updated to follow — this restores the correct contract.
  • Note: open PR Kafka Connect: Precompute UUID-as-bytes flag in RecordConverter #16654 ("Kafka Connect: Precompute UUID-as-bytes flag in RecordConverter") touches the same method but explicitly preserves the current byte[] behavior, so it does not fix this bug; whichever of the two merges first, the other will need a rebase.

Testing done

  • Updated TestRecordConverter#testUUIDConversionWithParquet to assert the field equals the original UUID, replacing the UUIDUtil.convert(UUID_VAL) byte[] expectation.
  • ./gradlew :iceberg-kafka-connect:iceberg-kafka-connect:check passes — TestRecordConverter 59/59, full module 122/122, 0 failures.

AI Disclosure

  • Tool: Claude Code — used to analyze the code, implement the fix, and update the test.

RecordConverter.convertUUID() converted UUID values to byte[] when the
target file format is Parquet. The Parquet UUID writer (ParquetValueWriters.uuids())
expects a java.util.UUID and converts to bytes internally, so writing a
UUID column with the default file format threw ClassCastException: class
[B cannot be cast to class java.util.UUID.

The byte[] branch matched the writer contract before apache#11904 changed
ParquetValueWriters' UUID writer to accept UUID directly; kafka-connect
was not updated to follow. This removes the byte[] conversion so
convertUUID always returns a UUID, matching ORC/Avro and the current
Parquet writer contract.

Generated-by: Claude Code
@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Aug 21, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions Bot closed this Aug 28, 2026
@vbhanuchander-lang

Copy link
Copy Markdown
Contributor

@thswlsqls same as #17080 — the stale bot closed this on 28 Aug without anyone reviewing it, and
the bug is still on main. I reproduced the failure and confirmed your fix resolves it.

Reproduced on main (8ea7d00, 3 Sep). Converting a UUID column with
write.format.default=parquet and handing the result to the Parquet writer:

>>> convertUUID returned: [B
java.lang.ClassCastException: class [B cannot be cast to class java.util.UUID

With your patch applied, the same path gives:

>>> convertUUID returned: java.util.UUID
>>> parquet write OK, bytes=500

Your diagnosis of why is right. ParquetValueWriters.uuids() returns a
PrimitiveWriter<UUID> and does the byte conversion itself, so the byte[] branch in
convertUUID() hands it the wrong type. The branch matched the writer contract until #11904
changed it; kafka-connect was not updated to follow. ORC and Avro were never affected because
that branch only fired for Parquet, which is also why removing it cannot regress them.

The full iceberg-kafka-connect module is green with the patch, 134 tests.

One thing a reviewer should know, since it is not obvious from the thread: the alternative approach
in #16654 (precompute the UUID-as-bytes flag) was closed unmerged on 8 Aug, so this is the only live
proposal for #17076.

I would add that testUUIDConversionWithParquet as it stands only asserts what the converter
returns, so it passes on a converter that still cannot write. A test that actually appends through
Parquet.write(...) would pin the contract that broke here — I am happy to hand you the one I used
above if you want it, or leave it as a follow-up.

@bryanck could this be reopened? The change is @thswlsqls's.

@thswlsqls
thswlsqls deleted the fix/kafka-connect-uuid-parquet-conversion branch September 9, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Kafka Connect: UUID columns fail to write with Parquet (ClassCastException)

2 participants