Skip to content

Fail to read INT96Timestamp from a list #3115

Description

@pratyush84

Describe the bug, including details regarding any error messages, version, and platform.

Can be reproduced through the following code -

 public void testIsElementTypeInt96Element(){
Configuration configuration = new Configuration();
configuration.setBoolean(READ_INT96_AS_FIXED, true);
MessageType parquetSchema = MessageTypeParser.parseMessageType( "message SchemaWithInt96 {\n"
+ " optional group list (LIST) {\n"
+ " repeated group list {\n"
+ " optional int96 a_timestamp;\n"
+ " }\n"
+ " }\n"
+ "}");
Schema avroSchema = new AvroSchemaConverter(configuration).convert(parquetSchema);
Assert.assertFalse(AvroRecordConverter.isElementType(
parquetSchema.getType("list").asGroupType().getType("list"),
AvroSchemaConverter.getNonNull(avroSchema.getFields().get(0).schema()).getElementType()
));
}

When trying to read a List of INT96 timestamps, it gives the following error-
Exception in thread "main" java.lang.IllegalArgumentException: INT96 is deprecated. As interim enable READ_INT96_AS_FIXED flag to read as byte array.

The issue is because in the AvroRecordConverter.java class, AvroSchemaConverter is instantiated without the ParquetConfiguration.
private static final AvroSchemaConverter CONVERTER = new AvroSchemaConverter(true);

Component(s)

Avro

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions