Skip to content

Remove dev dependencies before provider YAML check - #60709

Closed
dhanyabad11 wants to merge 1 commit into
apache:mainfrom
dhanyabad11:fix/remove-dev-deps-before-provider-yaml-check
Closed

Remove dev dependencies before provider YAML check#60709
dhanyabad11 wants to merge 1 commit into
apache:mainfrom
dhanyabad11:fix/remove-dev-deps-before-provider-yaml-check

Conversation

@dhanyabad11

Copy link
Copy Markdown
Contributor

This change adds a call to uv sync --no-dev --all-packages before running the provider YAML validation checks.

Why is this needed?

Currently, the run_provider_yaml_files_check.py script runs with all development dependencies installed, which may mask issues where:

  • Provider code requires dependencies that should be optional
  • Cross-provider dependencies are not properly marked as optional

By running uv sync --no-dev before validation, we remove dev dependencies and can detect cases where provider code has unhandled optional dependencies.

What this PR does:

  • Adds a remove_dev_dependencies() function that runs uv sync --no-dev --all-packages
  • Calls this function at the start of the main block, before ProvidersManager().initialize_providers_configuration()

Fixes: #60662

CopilotAI review requested due to automatic review settings January 17, 2026 15:15
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a step to remove development dependencies before running provider YAML validation checks. The change helps detect cases where provider code has unhandled optional dependencies that are normally masked when dev dependencies are present.

Changes:

  • Added remove_dev_dependencies() function that runs uv sync --no-dev --all-packages
  • Integrated the function call at the start of the main execution block

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +737 to +738
console.print(f"[red]Failed to remove dev dependencies: {result.stderr}[/]")
sys.exit(1)

CopilotAIJan 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The subprocess output (stdout) is captured but never displayed to the user, even on success. Consider either:

  1. Setting capture_output=False to let the output stream directly to the console (like in run_prepare_airflow_distributions.py), or
  2. Displaying result.stdout on success to show what was removed

This would provide better visibility into what the uv sync command did, which could be helpful for debugging.

Suggested change
console.print(f"[red]Failed to remove dev dependencies: {result.stderr}[/]")
sys.exit(1)
console.print(f"[red]Failed to remove dev dependencies: {result.stderr}[/]")
ifresult.stdout:
console.print(f"[red]uv sync stdout:[/]\n{result.stdout}")
sys.exit(1)
ifresult.stdout:
console.print(result.stdout)

Copilot uses AI. Check for mistakes.
@potiukpotiuk added all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs full tests needed We need to run full set of tests for this PR to merge labels Jan 17, 2026
@potiuk
potiukforce-pushed the fix/remove-dev-deps-before-provider-yaml-check branch from fcf10f9 to 2d8a004CompareJanuary 17, 2026 22:37
This change runs 'uv sync --no-dev --all-packages' before running the
provider YAML validation checks inside the breeze container. This ensures
that dev dependencies are removed so we can detect cases where provider
code has optional cross-provider dependencies that aren't handled properly.
The uv sync command is now run from the prek hook (check_provider_yaml_files.py)
before invoking the actual provider yaml check script.
Fixes: apache#60662
@dhanyabad11
dhanyabad11force-pushed the fix/remove-dev-deps-before-provider-yaml-check branch from 2d8a004 to f03e92aCompareJanuary 18, 2026 07:50
@dhanyabad11

Copy link
Copy Markdown
ContributorAuthor

@potiuk

@potiuk

Copy link
Copy Markdown
Member

Let's see if CI agrees.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR.
However, it seems like duplicated of #60728.

@potiuk

Copy link
Copy Markdown
Member

Yep.

@potiukpotiuk closed this Feb 16, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

all versionsIf set, the CI build will be forced to use all versions of Python/K8S/DBsarea:dev-toolsfull tests neededWe need to run full set of tests for this PR to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove all dev dependencies before provider yaml check

4 participants

@dhanyabad11@potiuk@jason810496