Uh oh!
There was an error while loading. Please reload this page.
Allow string conversion to datetime for since_time in KPO logs - #65498
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends KubernetesPodOperator/PodManager log fetching to handle since_time values provided as ISO-8601 strings by converting them into datetime objects instead of silently falling back to None.
Changes:
- Convert string
since_timevalues to datetimes inPodManager.fetch_container_logs()andKubernetesPodOperator._write_logs(). - Expand unit test coverage for valid datetime, invalid input, and ISO-string
since_timehandling. - Refactor the PodManager since-time test to use
time_machinerather than patchingpendulum.now.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py | Adds parsing of string since_time and broadens exception handling. |
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/pod.py | Adds parsing of string since_time in _write_logs and catches ValueError. |
providers/cncf/kubernetes/tests/unit/cncf/kubernetes/utils/test_pod_manager.py | Adds/updates tests for datetime, invalid, and string since_time scenarios. |
providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py | Adds a new _write_logs test for string since_time and adjusts invalid-case test wording. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
jscheffl
left a comment
There was a problem hiding this comment.
Cool, thanks for the improvement and rework!
Uh oh!
There was an error while loading. Please reload this page.
…e#65498) * Allow string conversion to datetime for since_time in KPO logs * Implement PR feedback
The since_seconds attribute in consume_logs (from class PodManager) and _write_logs (from class KubernetesPodOperator) is converted to None if since_time is not a datetime object. I have introduced code to check whether since_time is a string and convert it to a datetime object if possible instead of None.
This was suggested in PR #65232
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.