Skip to content

Change default .airflowignore syntax to glob - #42436

Merged
uranusjr merged 2 commits into
apache:mainfrom
astronomer:airflow-ignore-glob
Sep 30, 2024
Merged

Change default .airflowignore syntax to glob#42436
uranusjr merged 2 commits into
apache:mainfrom
astronomer:airflow-ignore-glob

Conversation

@uranusjr

Copy link
Copy Markdown
Member

This matches the ignore file behavior of many popular tools (e.g. Git, Docker) and is likely what most users expect without reading the documentation.

The documentation is also tweaked to emphasize the new default. I also decided to get rid of some regexp examples entirely; I figure most of the users are better off either using glob, or just customize the might_contain_dag_callable hook instead. The regexp syntax parser uses re2, which is quite limiting for security reasons, and does not offer significant advantages over glob for almost all use cases.

@uranusjruranusjr added airflow3.0:candidate Potential candidates for Airflow 3.0 airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes labels Sep 24, 2024
@uranusjr
uranusjr marked this pull request as ready for review September 24, 2024 08:31
@uranusjr
uranusjrforce-pushed the airflow-ignore-glob branch 2 times, most recently from 91540fa to 41d63ecCompareSeptember 25, 2024 02:00
This matches the ignore file behavior of many popular tools
(e.g. Git, Docker) and is likely what most users expect without reading
the documentation.
The documentation is also tweaked to emphasize the new default. I also
decided to get rid of some regexp examples entirely; I figure most of
the users are better off either using glob, or just customize the
might_contain_dag_callable hook instead. The regexp syntax parser uses
re2, which is quite limiting for security reasons, and does not offer
significant advantages over glob for almost all use cases.
@shahar1shahar1 added this to the Airflow 3.0.0 milestone Sep 27, 2024

@shahar1shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, small nit

Comment threaddocs/apache-airflow/core-concepts/dags.rst Outdated
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
@uranusjr
uranusjr merged commit 2eecca1 into apache:mainSep 30, 2024
@uranusjr
uranusjr deleted the airflow-ignore-glob branch September 30, 2024 04:43
joaopamaral pushed a commit to joaopamaral/airflow that referenced this pull request Oct 21, 2024
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
@Lee-WLee-W mentioned this pull request Nov 18, 2024
2 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

airflow3.0:breakingCandidates for Airflow 3.0 that contain breaking changesairflow3.0:candidatePotential candidates for Airflow 3.0kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@uranusjr@hussein-awala@shahar1