Skip to content

Split out datafusion-substrait and datafusion-proto CI feature checks, increase coverage - #15156

Merged
alamb merged 6 commits into
apache:mainfrom
alamb:alamb/check_features
Mar 12, 2025
Merged

Split out datafusion-substrait and datafusion-proto CI feature checks, increase coverage#15156
alamb merged 6 commits into
apache:mainfrom
alamb:alamb/check_features

Conversation

@alamb

@alambalamb commented Mar 11, 2025

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The coverage for feature flags needs to be improved, as explained on #15155

What changes are included in this PR?

  1. Move the checks for datafusion-substrait and datafusion-proto into their own jobs
  2. Consolidate the 'no default features' check into the per-crate checks

Note I will make a follow on PR to add additional coverage for flags in datafusion-functions and datafusion but I am trying to keep this PR reasonably sized

Are these changes tested?

Are there any user-facing changes?

@github-actionsgithub-actionsBot added the development-process Related to development process of DataFusion label Mar 11, 2025
@alambalamb changed the title Better CI feature checksSplit out datafusion-substrait and datafusion-proto CI feature checksMar 11, 2025
#
# Ensure via `cargo check` that the crate can be built with a
# subset of the features packages enabled.
linux-datafusion-common-features:

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diff makes of hard to read but I just moved what jobs each command was run in -- the overall coverage is the same or better

rust-version: stable
- name: Check datafusion-substrait (no-default-features)
run: cargo check --profile ci --all-targets --no-default-features -p datafusion-substrait
- name: Check datafusion-substrait (physical)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this adds coverage of compiling datafusion-substrait with the available feature flags

rust-version: stable
- name: Check datafusion-proto (no-default-features)
run: cargo check --profile ci --all-targets --no-default-features -p datafusion-proto
# fails due to https://github.com/apache/datafusion/issues/15157

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds coverage for the datafusion-proto crate, and in fact found a bug:

@alambalamb changed the title Split out datafusion-substrait and datafusion-proto CI feature checksSplit out datafusion-substrait and datafusion-proto CI feature checks, increase coverageMar 11, 2025
@alamb
alamb marked this pull request as ready for review March 11, 2025 14:57
@alamb
alamb requested a review from timsaucerMarch 11, 2025 14:57

@timsaucertimsaucer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks very reasonable to me and a good improvement. I left a note in the issue you raised that we will want to follow up on the commented out check.

@alamb

Copy link
Copy Markdown
ContributorAuthor

Thank you for the review @timsaucer !

@alambalamb changed the title Split out datafusion-substrait and datafusion-proto CI feature checks, increase coverageMinor: Split out datafusion-substrait and datafusion-proto CI feature checks, increase coverageMar 11, 2025
@alambalamb changed the title Minor: Split out datafusion-substrait and datafusion-proto CI feature checks, increase coverageSplit out datafusion-substrait and datafusion-proto CI feature checks, increase coverageMar 11, 2025
@alamb
alamb merged commit 592fe6a into apache:mainMar 12, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-processRelated to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@alamb@timsaucer