Skip to content

GH-2836: Support reading pure parquet files with cat - #3332

Merged
gszadovszky merged 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/fix-cat1
Sep 29, 2025
Merged

GH-2836: Support reading pure parquet files with cat#3332
gszadovszky merged 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/fix-cat1

Conversation

@ArnavBalyan

@ArnavBalyanArnavBalyan commented Sep 25, 2025

Copy link
Copy Markdown
Member

Rationale for this change

  • Parquet cat has a bug where it fails for files with hyphens since cat uses avro reader by default which has stricter rules.
  • Ensure we can still read pure parquet files, with parquet group reader as fallback.
  • Existing reader remains unchanged, we just read with parquet group reader if the read fails.
  • Closes: Problem with a cat #2836

Before:

Time elapsed: 1.170 s <<< ERROR!
org.apache.avro.SchemaParseException: Illegal character in: customer-name
at org.apache.avro.Schema.validateName(Schema.java:1625)
at org.apache.avro.Schema.access$400(Schema.java:94)
at org.apache.avro.Schema$Field.<init>(Schema.java:558)
at org.apache.avro.SchemaBuilder$FieldBuilder.completeField(SchemaBuilder.java:2258)
at org.apache.avro.SchemaBuilder$FieldBuilder.completeField(SchemaBuilder.java:2254)
at org.apache.avro.SchemaBuilder$FieldBuilder.access$5100(SchemaBuilder.java:2150)
at org.apache.avro.SchemaBuilder$GenericDefault.noDefault(SchemaBuilder.java:2557)
at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:376)

After:

order_id: 1001
customer-name: John Smith
product-category: Electronics
sale-amount: 299.99
region: North

Are these changes tested?

  • Yes

Are there any user-facing changes?

  • Yes

@ArnavBalyanArnavBalyan changed the title GH-2836: Support parquet only files in cat as fallbackGH-2836: Support reading pure parquet files with catSep 25, 2025
@ArnavBalyan

Copy link
Copy Markdown
MemberAuthor

cc @gszadovszky :) thanks!

@gszadovszky
gszadovszky merged commit 7e7a658 into apache:masterSep 29, 2025
7 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem with a cat

2 participants

@ArnavBalyan@gszadovszky