Add tarfile.TarPath

[FFY00]

BPO	45649
Nosy	@jaraco, @ethanfurman, @FFY00, @barneygale

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-10-28.15:59:21.866>
labels = ['type-feature', 'library', '3.11']
title = 'Add tarinfo.Path'
updated_at = <Date 2022-01-02.00:01:26.196>
user = 'https://github.com/FFY00'

bugs.python.org fields:

activity = <Date 2022-01-02.00:01:26.196>
actor = 'barneygale'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-10-28.15:59:21.866>
creator = 'FFY00'
dependencies = []
files = []
hgrepos = []
issue_num = 45649
keywords = []
message_count = 6.0
messages = ['405194', '405206', '405210', '409479', '409481', '409482']
nosy_count = 4.0
nosy_names = ['jaraco', 'ethan.furman', 'FFY00', 'barneygale']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45649'
versions = ['Python 3.11']

Linked PRs

• GH-89812: Churn pathlib.Path methods #104243
• [WIP] GH-89812: Add tarfile.TarPath #104272
• GH-89812: Add pathlib._PurePathExt #104810
• GH-89812: Clean up pathlib tests. #104829
• GH-89812: Churn pathlib.Path test methods #105807
• GH-89812: Add pathlib.UnsupportedOperation #105926
• GH-89812: Make symlink support configurable in pathlib tests. #106060
• GH-89812: Simplify creation of symlinks in pathlib tests. #106061
• GH-89812: Test that pathlib.Path.is_junction() returns false #106062
• GH-89812: Miscellaneous pathlib test improvements #106063
• GH-89812: Improve test for pathlib.Path.stat() #106064
• GH-89812: Add pathlib._PathBase #106337

[FFY00]

It would be helpful to have a pathlib-compatible object in tarfile, similarly to zipfile.Path.

[jaraco]

I vaguely recall exploring this concept and finding that tarfiles don’t supply the requisite interface because they’re not random access. I’m only 10% confident in that recollection, so worth exploring.

[FFY00]

That is good to know. This isn't very high on my priority list, but I will try to explore when I have some time.

[barneygale]

It's possible to do, but will be a little slow due to the nature of tar files. They're a big linked list of files, so you need to do a bunch of reads/seeks from the start to the end to enumerate all files.

I'd ask that we try to get bpo-24132 solved first. That would let us write:

 # tarfile.py
 class Path(pathlib.AbstractPath):
 def iterdir(self):
 ...
 def stat(self):
 ...

We'd fill in a smallish number of abstract methods to get a full Path-compatible class with read_text(), is_symlink() etc methods.

[jaraco]

I'd recommend not to block on bpo-24132. It's not obvious to me that subclassing would be valuable. It depends on how it's implemented, but in my experience, zipfile.Path doesn't and cannot implement the full interface of pathlib.Path. Instead zipfile.Path attempts to implement a protocol. At the time, the protocol was undefined, but now there exists importlib.resources.abc.Traversable (https://docs.python.org/3/library/importlib.html#importlib.abc.Traversable), the interface needed by importlib.resources. I'd honestly just create a standalone class, see if it can implement Traversable, and only then consider if it should implement a more complicated interface (such as something with symlink support or perhaps even later subclassing from pathlib.Path).

[barneygale]

If you're only aiming for Traversable compatibility, sure.

The original bug description asks for something that's pathlib-compatible and similar to zipfile.Path, which goes beyond the Traversable interface in attempting to emulate pathlib.Path.

The pathlib.Path interface is a good one - I see no reason it can't apply to zip and tar archives in full. Methods of Path objects already raise NotImplementedError if operations aren't supported (e.g. creating symlinks)

Some prototyping from a couple years back, including a tar path implementation: https://github.com/barneygale/pathlab/tree/master/pathlab

[encukou]

Anyone implementing this should be aware that tar members don't have unique names.
A path should generally refer to the last member of the given name (which would overwrite previous ones when extracting -- unless the previous members do shenanigans with symlinks/directories, but API for accessing individual members isn't expected to take that into account).
TarFile API like getmember will take care of most of this, but not all: for example, a “listdir” operation should probably de-duplicate its result.