Backslash in f-string format specifier gets escape

[domdfcoding]

Bug report

Between Python 3.12 alpha 7 and beta 1 the behaviour of format specifiers containing a backslash (e.g. \n, \u2603) in f-strings changed.

Previously, the __format__ method would get e.g. the newline character, but it now receives a backslash and an n character (i.e. the string "\n")

The following script demonstrates this:

# issue.py

class F:
	def __format__(self, format_spec):
		print("spec:", repr(format_spec))
		return format_spec.join(["a", "b", "c"])

print("out: ", repr((f"{F():\n}")))
print("out: ", repr((f"{F():\u2603}")))

print("out: ", repr((format(F(), "\n"))))
print("out: ", repr((format(F(), "\u2603"))))

$ python3.11 issue.py
spec: '\n'
out: 'a\nb\nc'
spec: '☃'
out: 'a☃b☃c'
spec: '\n'
out: 'a\nb\nc'
spec: '☃'
out: 'a☃b☃c'

$ python3.12 issue.py
spec: '\\n'
out: 'a\\nb\\nc'
spec: '\\u2603'
out: 'a\\u2603b\\u2603c'
spec: '\n'
out: 'a\nb\nc'
spec: '☃'
out: 'a☃b☃c'

In both cases the format() function works correctly - I only encounter the issue with f-strings.

I suspect this is related to the PEP 701 changes to f-strings, but I couldn't find anything in the discussion of that PEP to suggest this was a deliberate change.

Your environment

• CPython versions tested on: Python 3.11.2, 3.12.0alpha7, 3.12.0beta1
• Operating system and architecture: Ubuntu 20.04 (AMD64)

Linked PRs

• gh-105194: Fix format specifier escaped characters in f-strings #105231
• [3.12] gh-105194: Fix format specifier escaped characters in f-strings (GH-105231) #105234

[pablogsal]

@isidentical @lysnikolaou can you take this one?

[lysnikolaou]

Sure! Will probably have a look tomorrow morning.

[pablogsal]

@lysnikolaou I gave this a go and I think I have it. Can you check #105231 ?