Enable SQLAlchemy connection pool settings for file-based SQLite

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[jason810496]

Why

SQLAlchemy 2.0+ uses QueuePool by default for file-based SQLite databases, but Airflow unconditionally skips all pool configuration (pool_size, max_overflow, pool_recycle, pool_pre_ping) for any SQLite connection — a guard that was only correct for SQLAlchemy 1.x.

What

• Remove the blanket not SQL_ALCHEMY_CONN.startswith("sqlite") guard in prepare_engine_args and replace it with a targeted check that only skips pool settings for in-memory SQLite (sqlite://, sqlite:///:memory:), which uses SingletonThreadPool that doesn't support max_overflow.
• File-based SQLite now receives the same pool configuration as PostgreSQL and MySQL.
• Update the SQLAlchemy docs link from 1.4 to 2.0.
• Add unit tests covering both cases:
  • File-based SQLite gets pool settings through configure_orm.
  • In-memory SQLite skips pool settings (tested via prepare_engine_args directly, since configure_orm rejects sqlite:///:memory: as a relative path before reaching pool logic).

[Copilot]

Pull request overview

This PR updates Airflow’s SQLAlchemy engine configuration so that connection pool settings are applied to file-based SQLite connections (matching SQLAlchemy 2.0+ defaults), while still skipping pool settings for in-memory SQLite connections where they are not supported.

Changes:

• Adjust prepare_engine_args to skip pool settings only for in-memory SQLite instead of all SQLite.
• Update the SQLAlchemy docs link to the 2.0 documentation.
• Add unit tests to validate pool settings for file-based SQLite and skipping behavior for in-memory SQLite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
airflow-core/src/airflow/settings.py	Refines pool-setting guard for SQLite and updates SQLAlchemy docs link.
airflow-core/tests/unit/core/test_sqlalchemy_config.py	Adds tests for file-based SQLite pooling and in-memory SQLite pool-setting skip behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[jason810496]

The only failing static check in CI was fixed in latest main, so I will merge it.

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test