Skip to content

Update arrow/parquet to arrow/parquet 53.3.0 - #13508

Merged
jonahgao merged 3 commits into
apache:mainfrom
alamb:alamb/update_to_arrow
Nov 21, 2024
Merged

Update arrow/parquet to arrow/parquet 53.3.0#13508
jonahgao merged 3 commits into
apache:mainfrom
alamb:alamb/update_to_arrow

Conversation

@alamb

@alambalamb commented Nov 20, 2024

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

There are some other fixes waiting on this arrow release so let's get them integrated into DataFusion

What changes are included in this PR?

Update to latest arrow/parquet

Are these changes tested?

By CI

Are there any user-facing changes?

@alamb
alamb marked this pull request as ready for review November 20, 2024 22:57
@jayzhan211

jayzhan211 commented Nov 21, 2024

Copy link
Copy Markdown
Contributor

Why is CI forced to run in arrow 53.3 🤔 ?

@jayzhan211

This comment was marked as outdated.

Sign in to view
@jayzhan211
fix ci test
1b8bec5
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
@github-actionsgithub-actionsBot added the sqllogictest SQL Logic Tests (.slt) label Nov 21, 2024
@jayzhan211

jayzhan211 commented Nov 21, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

I push the fix since this is blocking CI.

Since #13291 looks like in discussion, so I fix up the Like test which is consistent with Postgres. We can revert this to old result if there is consensus

@jonahgao

jonahgao commented Nov 21, 2024

Copy link
Copy Markdown
Member

Thanks @alamb@jayzhan211

@jonahgao
jonahgao merged commit a2811fc into apache:mainNov 21, 2024
@jonahgao

jonahgao commented Nov 21, 2024

Copy link
Copy Markdown
Member

Why is CI forced to run in arrow 53.3 🤔 ?

It seems that CI uses the latest version of dependencies that are semver compatible

@alamb

alamb commented Nov 21, 2024

Copy link
Copy Markdown
ContributorAuthor

Why is CI forced to run in arrow 53.3 🤔 ?

My thinking was that explicitly using arrow 53.3 will ensure that any bugs we fix that rely on arrow 53.3 features will work correctly.

If we don't update DataFusion to explicitly use 53.3.0 in Cargo.toml, CI will run (and use 53.3.0) but other projects with DataFusion could potentially use 53.2.0. If we add code to DataFusion that relies on 53.3.0 (like some of the LIKE fixes) this would be broken

@alamb
alamb deleted the alamb/update_to_arrow branch November 21, 2024 12:30
@alamb

alamb commented Nov 21, 2024

Copy link
Copy Markdown
ContributorAuthor

Thanks @jayzhan211

@findepifindepi mentioned this pull request Nov 23, 2024
Merged
findepi
findepi reviewed Nov 23, 2024
View reviewed changes
Comment threaddatafusion/sqllogictest/test_files/string/string_view.slt
@@ -39,8 +39,19 @@ drop table test_source

# TODO: Revisit this issue after upgrading to the arrow-rs version that includes apache/arrow-rs#6671.

@findepifindepiNov 23, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This became now addressable - #13538

wiedld pushed a commit to influxdata/arrow-datafusion that referenced this pull request Jan 7, 2025
wiedld
Revert "Update arrow/parquet to arrow/parquet53.3.0(apache#13508)"
e607983
This reverts commit a2811fc.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@findepifindepifindepi left review comments
@jonahgaojonahgaojonahgao approved these changes

Assignees

No one assigned

Labels

sqllogictestSQL Logic Tests (.slt)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@alamb@jayzhan211@jonahgao@findepi