Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Fix module loading in logging config - #54555

Merged
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading
Aug 19, 2025
Merged

Fix module loading in logging config#54555
potiuk merged 13 commits into
apache:mainfrom
viiccwen:fix-module-loading

Conversation

@viiccwen

@viiccwenviiccwen commented Aug 15, 2025

Copy link
Copy Markdown
Contributor

Summary

Fix issue where custom logging configuration with simple module paths (like log_config.LOGGING_CONFIG) caused triggerer and scheduler to fail during startup.

Problem

When users set logging_config_class = log_config.LOGGING_CONFIG in their airflow.cfg, the load_logging_config() function would fail because:

  1. logging_class_path.rsplit(".", 1)[0] produces "log_config"
  2. import_string("log_config") fails because it expects a dot-separated path like "module.submodule.attribute"
  3. This caused ImportError and prevented REMOTE_TASK_LOG and DEFAULT_REMOTE_CONN_ID from being loaded
  4. In some environments, this prevented triggerer and scheduler from starting

Solution

Replace import_string() with importlib.import_module() in load_logging_config()

Changes

Code

  • inairflow-core/src/airflow/logging_config.py, withload_logging_config()

Test

  • in airflow-core/tests/unit/core/test_logging_config.py
  • Added: Parametrized test for different module path structures
  • Added: Test for graceful fallback when remote logging vars are missing
  • Added: Helper method to reduce test code duplication

Related Issues

resolves#54414

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
@jason810496
jason810496 self-requested a review August 16, 2025 03:06
@viiccwen

Copy link
Copy Markdown
ContributorAuthor

Hello @jason810496 ,
Could you help me to review?
Thx!

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the PR! LGTM overall.

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.

@jason810496jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Comment threadairflow-core/src/airflow/logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
@jason810496
jason810496 requested a review from Lee-WAugust 19, 2025 09:36
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
Comment threadairflow-core/tests/unit/core/test_logging_config.py Outdated
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
@Lee-W

Copy link
Copy Markdown
Member

I think we're good to merge once CI passes 🙂

@potiuk
potiuk merged commit 462ce9a into apache:mainAug 19, 2025
58 checks passed
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actionsBot pushed a commit that referenced this pull request Aug 19, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

StatusBranchResult
v3-0-testPR Link

potiuk pushed a commit that referenced this pull request Aug 20, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
(cherry picked from commit 462ce9a)
Co-authored-by: Vic Wen <99777196+viiccwen@users.noreply.github.com>
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* fix(logging): fix module loading by using `import_module()`
- Replace `import_string()` with `import_module()` in `load_logging_config()`
- Fix issue where simple module paths like `log_config.LOGGING_CONFIG` failed
- Add `importlib` import inside function to avoid circular imports
* test(logging): add tests for logging config module path handling
- Add parametrized test for simple and nested module paths
- Add test for graceful fallback when remote logging vars are missing
- Create helper method to reduce test code duplication
- Test both log_config.LOGGING_CONFIG and nested.module.LOGGING_CONFIG scenarios
Ensures logging config works with various module path structures
* refactor(logging): move `import_module` to the top level
* test(logging): optimize test constants to reduce code duplication
Replace `SETTINGS_FILE_NESTED_MODULE` replicate part with string replacement from `SETTINGS_FILE_SIMPLE_MODULE`
* test(logging): use shared helper method for logging config validation
Replace manual assertions with `self._verify_basic_logging_config` in fallback test to ensure consistent validation across all logging config tests.
* refactor(logging_config): simplify, improve test config vars
- Remove redundant `SETTINGS_FILE_NESTED_MODULE`
- Rename `SETTINGS_FILE_SIMPLE_MODULE` to `SETTINGS_FILE_WITH_REMOTE_VARS`
* refactor(logging): imporve & simplify logging testing file
1. remove `SETTING_FILE_NO_REMOTE_VARS` bcz redundant var
2. add return typo with `_verify_basic_logging_config`
3. add unittest.mock `call`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Altered mechanism attempts to read REMOTE_TASK_LOG from incorrect location

5 participants

@viiccwen@Lee-W@jason810496@prdai@potiuk