Skip to content

Built-in conversions for nested logical types fail on Avro <= 1.8 #3114

Description

@clairemcginty

Describe the bug, including details regarding any error messages, version, and platform.

When an Avro record contains nested logical types but no top-level logical types, i.e.

{
"type": "record",
"name": "NestedLogicalTypes",
"fields": [
{
"name": "innerType",
"type": {
"type": "record",
"name": "InnerType",
"fields": [
{
"name": "ts",
"type": {
"type": "long",
"logicalType": "timestamp-millis"
}
}
]
}
}
]
}

Writes fails to convert the logical types from in-memory representation to underlying primitive:

Cause: java.lang.ClassCastException: class org.joda.time.DateTime cannot be cast to class java.lang.Number (org.joda.time.DateTime is in unnamed module of loader 'app'; java.lang.Number is in module java.base of loader 'bootstrap')
at org.apache.parquet.avro.AvroWriteSupport.writeValueWithoutConversion(AvroWriteSupport.java:345)
at org.apache.parquet.avro.AvroWriteSupport.writeValue(AvroWriteSupport.java:282)
at org.apache.parquet.avro.AvroWriteSupport.writeRecordFields(AvroWriteSupport.java:202)
at org.apache.parquet.avro.AvroWriteSupport.writeRecord(AvroWriteSupport.java:185)
at org.apache.parquet.avro.AvroWriteSupport.writeValueWithoutConversion(AvroWriteSupport.java:371)
at org.apache.parquet.avro.AvroWriteSupport.writeValue(AvroWriteSupport.java:285)
at org.apache.parquet.avro.AvroWriteSupport.writeRecordFields(AvroWriteSupport.java:202)
at org.apache.parquet.avro.AvroWriteSupport.write(AvroWriteSupport.java:178)
at org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:152)
at org.apache.parquet.hadoop.ParquetWriter.write(ParquetWriter.java:428)

Component(s)

Avro

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions