Fix ObjectStoragePath NoCredentialsError when using conn_id with remote stores

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

[vatsrahul1001]

Problem

After the migration from CloudPath to ProxyUPath (#60519), ObjectStoragePath
silently breaks for any remote store when a conn_id is provided.

path=ObjectStoragePath("s3://my-bucket/prefix", conn_id="my_aws_conn")
path.exists() # NoCredentialsErrorpath.mkdir() # NoCredentialsErrorpath.iterdir() # NoCredentialsError

Root cause: ProxyUPath delegates all filesystem operations to an inner self.wrapped UPath instance. That instance is constructed with empty storage_options because conn_id is stripped before super().init() (per #62701 to prevent it leaking into fsspec). So self.wrapped.fs creates an unauthenticated s3fs/gcsfs — ignoring the conn_id entirely.

This affects every delegated method: exists, mkdir, is_dir, is_file, touch, unlink, rmdir, iterdir, glob, rglob, walk, rename, read_bytes, write_bytes, read_text, write_text, and open.

Reported by @pankajkoti in 3.2.0rc1 testing.

Fix
Inject the Airflow-authenticated filesystem directly into self.wrapped._fs_cached immediately after super().init().

UPath propagates _fs_cached from parent to child whenever new path segments are appended (p / "sub", parent, iterdir results, glob results, etc.), so two injection points cover every case:

init — for paths constructed directly
_from_upath — for paths wrapped from a fresh UPath with no cached fs
This fixes all delegated methods in 22 lines with no per-method overrides.

Testing

• closes: DAGs with ObjectStoragePath failing to locate credentials with Airflow 3.2 rc #64632
• related: #ISSUE

• Yes (please specify the tool below)
  claude

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[pankajkoti]

Thanks @vatsrahul1001 for the lightning-fast fix 🚀 . I applied and tested the patch, and my DAG is happy now ✅ .

It would be great if someone with deeper expertise could also review the PR for any nuances I might have missed.

[Copilot]

Pull request overview

Fixes ObjectStoragePath credential resolution regressions introduced by the move to ProxyUPath, ensuring delegated path operations use Airflow-attached/authenticated filesystems when conn_id is provided.

Changes:

• Injects the Airflow-authenticated filesystem into the wrapped UPath’s _fs_cached during ObjectStoragePath.__init__.
• Adds a secondary injection path in _from_upath for wrapping “fresh” UPath instances without a cached filesystem.
• Adds regression tests covering delegated operations (e.g. exists, mkdir, is_dir, touch, iterdir) with conn_id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
task-sdk/src/airflow/sdk/io/path.py	Injects authenticated FS into __wrapped__._fs_cached in __init__ and _from_upath to fix delegated method behavior with conn_id.
task-sdk/tests/task_sdk/io/test_path.py	Adds regression tests ensuring delegated filesystem operations use the Airflow-attached FS when conn_id is set (including propagation to children).

[kaxil]

left few comments

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test