Add Python 3.13 support for Airflow.

[potiuk]

Added Python 3.13 support across the codebase, including:

• CI/CD workflow updates to handle Python 3.13.
• Dockerfile and environment variable adjustments for compatibility.
• Documentation updates to list Python 3.13 as supported.
• Dependency and code changes for compatibility with new/changed
  libraries (e.g., numpy 2.x, greenlet, pendulum).
• Test and plugin manager adjustments for Python 3.13-specific behaviors.
• Removal or skipping of tests and features not compatible with Python 3.13.
• Improved error handling and logging for unsupported plugins/providers
  on Python 3.13.
• Updated all provider README.rst files to:
  • List Python 3.13 as a supported version.
  • Adjust dependency requirements for Python 3.13 (e.g., pandas,
    pyarrow, ray, etc.).
  • Add or update notes about excluded or conditionally supported
    dependencies for Python 3.13.
  • Document any known incompatibilities or workarounds for specific
    providers.
• Updated all provider pyproject.toml files and automation to:
  • Add Python 3.13 to the supported classifiers.
  • Adjust requires-python and dependency constraints for Python 3.13.
  • Exclude Python 3.13 for providers that are not yet compatible (e.g.,
    those depending on Flask AppBuilder).
  • Add conditional dependencies for new versions of libraries required
    by Python 3.13.
• Updated Breeze and related scripts to:
• Build and test with Python 3.13.
• Update documentation and release scripts to include Python 3.13.
• Ensure all dev tooling and test environments work with Python 3.13.
• Removed remaining FAB related code from core and removed tests for
  older FAB provider versions (<1.3.0)
• some of the tests in core still depend on FAB - but we make them
  optional - both for parsing (we need to skip some imports) and
  execution - because they test a legacy behaviour of embedding
  Flask plugins that is still supported in core.
• k8s tests now run also with SimpleAuthManager and SimpleAuthManager
  is used automatically when runnin tests in Python 3.13
• migration tests had to be excluded fo Python 3.13 because we cannot
  migrate down on Python 3.13 without FAB tables created. The tests
  will be brought back when FAB is supported or when we release 3.1
  and start working on 3.2, we should be able to migrate down to 3.1
• Weaviate tests have been updated to handle case where weaviate-client
  is > 2.10.0 - because it started to use httpx instead of requests,
  and Python 3.13 actually unblocks using older version of weaviate
  (it was previously blocked indirectly via Flask dependencies and
  grpcio).
• Added doc change in our rules on what is the "default" Python version
  for airflow images - handling the case where we cannot use "newest"
  Python version as default where not all "regular" image providers are
  supported in the latest version
• Connection is now flushed in configure_git_connection_for_dag_bundle
  just before clear_db_connections - in order to handle Python 3.13
  case where FAB app is not initialized by other fixtures - performing
  implicit flush along the way
• Cleaning db connections was added to test_dag_run and test_dags test
  in fast_api as the test implicitly rely on having connections
  cleared (no test git connections added) - this will make the tests
  side-effect free.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[jscheffl]

Another review round. LIKE It. Special thanks for also adding notes to the selective areas where upper version binding is made. Hope we remember this in future.

Also I very much like the proposal to keep the non-versioned reference image with 3.12 and upgrade as soon as standard installed providers are all Python 3.13 capable. Especially having reference image w/o Fab would make it hardly usable and would confuse users.

I would be happy to get this merged.

Thanks for the efforts.

[amoghrajesh]

Some qns / nits on commit 2

[aritra24]

This is great Jarek! Had a minor nit, but other than that all looks good from a high level.

[potiuk]

Pushed changes trying out the comments in places I am not sure.

[gopidesupavan]

Thanks @potiuk for all the work, may iterations not an easy task .

[gopidesupavan]

#protm

[potiuk]

Aaand merged :)

[aritra24]

Long effort!! Great job!

[shahar1]

#protm for sure, great job Jarek! :)

[eladkal]

🎉 🎉🎉

[jscheffl]

Wohoo! Party!

[kyungjunleeme]

Cool!