From a08226c41e90eac851ec702cd4f463b80db08d7f Mon Sep 17 00:00:00 2001 From: Ash Berlin-Taylor Date: Tue, 16 Sep 2025 15:59:08 +0100 Subject: [PATCH 1/2] Add newsfragment to call out new features available now with structlog loggers Nothing _needs_ to change, but people (both DAG authors and provider authors) can now start to take advantage of this. --- .../newsfragments/52651.significant.rst | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 airflow-core/newsfragments/52651.significant.rst diff --git a/airflow-core/newsfragments/52651.significant.rst b/airflow-core/newsfragments/52651.significant.rst new file mode 100644 index 0000000000000..c2cdf9b7efc46 --- /dev/null +++ b/airflow-core/newsfragments/52651.significant.rst @@ -0,0 +1,53 @@ +Airflow now uses `https://www.structlog.org/en/stable/ `_ everywhere. + +Most users should not notice the difference, but it is now possible to emit structured log key/value pairs from tasks. + +If your class subclasses LoggingMixin (which all BaseHook and BaseOperator do -- i.e. all hooks and operators) then ``self.log`` is not a ` `_. + +The advantage of using structured logging is that it is much easier to find specific information about log message, especially when using a central store such as OpenSearch/Elastic/Splunk etc. You don't have to make any changes, but you can now take advantage of this. + +.. code-block:: python + + # Inside a Task/Hook etc. + + # Before: + # self.log.info("Registering adapter %r", item.name) + # Now: + self.log.info("Registering adapter", name=item.name) + +This will produce a log that (in the UI) will look something like this:: + + [2025-09-16 10:36:13] INFO - Registering adapter name="adapter1" + + +or in JSON (i.e. the log files on disk):: + + {"timestamp": "2025-09-16T10:36:13Z", "log_level": "info", "event": "Registering adapter", "name": "adapter1"} + + +You can also use structlog loggers at the top level of modules etc, and stdlib both continue to work: + +.. code-block:: python + + import logging + import structlog + + log1 = logging.getLogger(__name__) + log2 = strcutlog.get_logger(__name__) + + log1.info("Loading something from %s", __name__) + log2.info("Loading something", source=__name__) + +(You can't add arbitrary key/value pairs to stdlib, but the normal percent-formatter approaches still work fine.) + + +* Types of change + + * [ ] Dag changes + * [ ] Config changes + * [x] API changes + * [ ] CLI changes + * [ ] Behaviour changes + * [ ] Plugin changes + * [ ] Dependency changes + * [ ] Code interface changes From 48818274ffb59dc10f3e0ddb4f1a05bad32ac86d Mon Sep 17 00:00:00 2001 From: Kaxil Naik Date: Tue, 16 Sep 2025 17:36:42 +0100 Subject: [PATCH 2/2] Update airflow-core/newsfragments/52651.significant.rst --- airflow-core/newsfragments/52651.significant.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/airflow-core/newsfragments/52651.significant.rst b/airflow-core/newsfragments/52651.significant.rst index c2cdf9b7efc46..d71834af93827 100644 --- a/airflow-core/newsfragments/52651.significant.rst +++ b/airflow-core/newsfragments/52651.significant.rst @@ -2,7 +2,7 @@ Airflow now uses `https://www.structlog.org/en/stable/ `_ everywhere. Most users should not notice the difference, but it is now possible to emit structured log key/value pairs from tasks. -If your class subclasses LoggingMixin (which all BaseHook and BaseOperator do -- i.e. all hooks and operators) then ``self.log`` is not a ` `_. +If your class subclasses LoggingMixin (which all BaseHook and BaseOperator do -- i.e. all hooks and operators) then ``self.log`` is now a ` `_. The advantage of using structured logging is that it is much easier to find specific information about log message, especially when using a central store such as OpenSearch/Elastic/Splunk etc. You don't have to make any changes, but you can now take advantage of this.