os.stat() on Windows succeeds for nonexistent paths with trailing spaces

[laloch]

BPO	40238
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba, @gaborbernat, @laloch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-04-09.14:25:30.374>
labels = ['3.8', 'type-bug', 'library', '3.9', 'OS-windows']
title = 'os.stat() on Windows succeeds for nonexistent paths with trailing spaces'
updated_at = <Date 2020-04-10.13:57:20.426>
user = 'https://github.com/laloch'

bugs.python.org fields:

activity = <Date 2020-04-10.13:57:20.426>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2020-04-09.14:25:30.374>
creator = 'laloch'
dependencies = []
files = []
hgrepos = []
issue_num = 40238
keywords = []
message_count = 6.0
messages = ['366054', '366063', '366111', '366120', '366123', '366125']
nosy_count = 7.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'gaborjbernat', 'laloch']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40238'
versions = ['Python 3.8', 'Python 3.9']

Linked PRs

• gh-84419: Fix the execute permissions in os.stat() on Windows #155392

[laloch]

On Windows (Server 2012 R2 in my case) os.stat() seems to be striping significant trailing spaces off the path argument:

>>> import os
>>> os.stat("c:\\Program Files ")
os.stat_result(st_mode=16749, st_ino=281474976710717, st_dev=173025906, st_nlink=1, st_uid=0, st_gid=0, st_size=8192, st_atime=1586154685, st_mtime=1586154685, st_ctime=1377178576)
>>> os.stat("c:\\Program Files\\ ")
os.stat_result(st_mode=16749, st_ino=281474976710717, st_dev=173025906, st_nlink=1, st_uid=0, st_gid=0, st_size=8192, st_atime=1586154685, st_mtime=1586154685, st_ctime=1377178576)
>>> # consequently
>>> os.path.isdir("c:\\Program Files\\ ")
True
>>> os.path.isdir("c:\\Program Files ")
True
>>> os.scandir("c:\\Program Files ")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\Program Files '

The same also applies to regular files, not just directories.

[eryksun]

os.stat() seems to be striping significant trailing spaces off
the path argument

The Windows file API normalizes paths to replace forward slashes with backslashes; resolve relative paths and "." and ".." components; strip trailing spaces and dots from the final component; and map reserved DOS device names in the final component of non-UNC paths to device paths (e.g. "C:/Temp/con " -> r"\\.\con").

Use a "\\?\" extended device path to bypass normalization, e.g. r"\\?\C:\Program Files ". Because an extended path doesn't get normalized in an open or create context, it should only use backslash as the path separator and should be fully qualified, without any "." and ".." components. Also, a non-device UNC path has to explicitly use the "UNC" device, e.g. "//server/share/spam... " -> r"\\?\UNC\server\share\spam... ".

That said, I strongly advise against using an extended path to create or rename files to use reserved DOS device names or trailing dots and spaces. Such filenames will be inaccessible by most applications.