Uh oh!
There was an error while loading. Please reload this page.
Switch from xz2 to liblzma to reduce duplicate dependencies - #17509
Merged
Conversation
alamb
commented
Sep 10, 2025
Contributor
btw there is a bunch of work upstream to make an |
findepi
commented
Sep 11, 2025
Member
is this "fixes #15342" ? |
timsaucer
commented
Sep 11, 2025
MemberAuthor
Thank you @findepi . I should have searched the issues to find it reported. The upstream PR is merged, so once it's next release comes out I can update this PR and mark it ready. |
timsaucer
commented
Sep 11, 2025
MemberAuthor
It looks like upstream is on a 2 month release cadence, so we'll see if that lands first or the |
timsaucerforce-pushed
the
chore/single_bzip_dep
branch
from
November 17, 2025 17:08
1301f36 to
ab83896Comparetimsaucer
marked this pull request as ready for review
November 17, 2025 17:40
timsaucer
commented
Nov 17, 2025
MemberAuthor
@N-Boutaib@gabotechs ready for review |
findepi
approved these changes
Nov 17, 2025
Uh oh!
There was an error while loading. Please reload this page.
N-Boutaib pushed a commit
to DataDog/datafusion
that referenced
this pull request
Nov 18, 2025
…7509) Closesapache#15342 Reduces the duplicate dependencies. We currently depend on bzip2 in two different ways. In attempting to reduce this, I needed to update `async-compression` which caused two different libraries to link to the system lzma library. This is not allowed in rust. This PR updates avro-rs, but we cannot merge this PR until that crate merges apache/avro-rs#284 and we remove the crates.io patch this PR contains. Update avro-rs and switch from the unmaintained xz2 crate to liblzma. Unit tests. None. This is simply a dependency update to a more recent crate. (cherry picked from commit 5765a08)
N-Boutaib added a commit
to DataDog/datafusion
that referenced
this pull request
Nov 19, 2025
…5765a08 Switch from xz2 to liblzma to reduce duplicate dependencies (apache#17509)
ahmed-mez pushed a commit
to DataDog/datafusion
that referenced
this pull request
Nov 21, 2025
…7509) ## Which issue does this PR close? Closesapache#15342 ## Rationale for this change Reduces the duplicate dependencies. We currently depend on bzip2 in two different ways. In attempting to reduce this, I needed to update `async-compression` which caused two different libraries to link to the system lzma library. This is not allowed in rust. This PR updates avro-rs, but we cannot merge this PR until that crate merges apache/avro-rs#284 and we remove the crates.io patch this PR contains. ## What changes are included in this PR? Update avro-rs and switch from the unmaintained xz2 crate to liblzma. ## Are these changes tested? Unit tests. ## Are there any user-facing changes? None. This is simply a dependency update to a more recent crate.
logan-keede pushed a commit
to logan-keede/datafusion
that referenced
this pull request
Nov 23, 2025
…7509) ## Which issue does this PR close? Closesapache#15342 ## Rationale for this change Reduces the duplicate dependencies. We currently depend on bzip2 in two different ways. In attempting to reduce this, I needed to update `async-compression` which caused two different libraries to link to the system lzma library. This is not allowed in rust. This PR updates avro-rs, but we cannot merge this PR until that crate merges apache/avro-rs#284 and we remove the crates.io patch this PR contains. ## What changes are included in this PR? Update avro-rs and switch from the unmaintained xz2 crate to liblzma. ## Are these changes tested? Unit tests. ## Are there any user-facing changes? None. This is simply a dependency update to a more recent crate.
ChanTsune added a commit
to ChanTsune/datafusion
that referenced
this pull request
Jan 16, 2026
`Switch from xz2 to liblzma to reduce duplicate dependencies (apache#17509)` made it possible to compile to wasm (liblzma supports wasm), so enable the compression feature in wasmtest. Refs: apache#17509
ChanTsune added a commit
to ChanTsune/datafusion
that referenced
this pull request
Jan 17, 2026
`Switch from xz2 to liblzma to reduce duplicate dependencies (apache#17509)` made it possible to compile to wasm (liblzma supports wasm), so enable the compression feature in wasmtest. Refs: apache#17509
github-merge-queueBot
pushed a commit
that referenced
this pull request
Jan 21, 2026
## Which issue does this PR close? - Refs #17509 (Switch from xz2 to liblzma to reduce duplicate dependencies) ## Rationale for this change Switching from `xz2` to `liblzma` enabled wasm builds (liblzma supports wasm), so we can now enable the `compression` feature in wasm tests. ## What changes are included in this PR? - Add the `compression` feature to the `datafusion` dependency in `datafusion/wasmtest/Cargo.toml`. ## Are these changes tested? - Verified by CI in this PR. ## Are there any user-facing changes? - No. (test configuration change only)
de-bgunter pushed a commit
to de-bgunter/datafusion
that referenced
this pull request
Mar 24, 2026
## Which issue does this PR close? - Refs apache#17509 (Switch from xz2 to liblzma to reduce duplicate dependencies) ## Rationale for this change Switching from `xz2` to `liblzma` enabled wasm builds (liblzma supports wasm), so we can now enable the `compression` feature in wasm tests. ## What changes are included in this PR? - Add the `compression` feature to the `datafusion` dependency in `datafusion/wasmtest/Cargo.toml`. ## Are these changes tested? - Verified by CI in this PR. ## Are there any user-facing changes? - No. (test configuration change only)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes#15342
Rationale for this change
Reduces the duplicate dependencies.
We currently depend on bzip2 in two different ways. In attempting to reduce this, I needed to update
async-compressionwhich caused two different libraries to link to the system lzma library. This is not allowed in rust.This PR updates avro-rs, but we cannot merge this PR until that crate merges apache/avro-rs#284 and we remove the crates.io patch this PR contains.
What changes are included in this PR?
Update avro-rs and switch from the unmaintained xz2 crate to liblzma.
Are these changes tested?
Unit tests.
Are there any user-facing changes?
None. This is simply a dependency update to a more recent crate.