.replace functions in datetime do not call __new__

[joelgibson]

BPO	44876
Nosy	@joelgibson

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-08-09.22:58:31.112>
labels = ['type-bug', 'library']
title = '.replace functions in datetime do not call __new__'
updated_at = <Date 2021-08-09.22:58:31.112>
user = 'https://github.com/joelgibson'

bugs.python.org fields:

activity = <Date 2021-08-09.22:58:31.112>
actor = 'joelgibson'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-08-09.22:58:31.112>
creator = 'joelgibson'
dependencies = []
files = []
hgrepos = []
issue_num = 44876
keywords = []
message_count = 1.0
messages = ['399295']
nosy_count = 1.0
nosy_names = ['joelgibson']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44876'
versions = []

Linked PRs

• gh-89039: call subclass constructors in datetime.*.replace #114780
• [3.12] gh-89039: Call subclass constructors in datetime.*.replace (GH-114780) #131239

Rationale for not backporting: #89039 (comment).

[joelgibson]

I've come here while investigating a segfault when datetime.replace(...) was called with a Pandas Timestamp object:
pandas-dev/pandas#42305

The Python implementation of datetime.replace

cpython/Lib/datetime.py

Line 1823 in 03e5647

def replace(self, year=None, month=None, day=None, hour=None,

is polymorphic, in the sense that if a custom object like pd.Timestamp is passed in, a pd.Timestamp will come out rather than a datetime. This works just fine (copying and using Python code makes this segfault disappear).

The C implementation is also polymorphic

cpython/Modules/_datetimemodule.c

Line 5845 in 03e5647

datetime_replace(PyDateTime_DateTime *self, PyObject *args, PyObject *kw)

but I think that something in the C implementation is wrong: eventually tp_alloc gets called for the passed type, but never tp_new, and then afterwards the custom type is treated just like a regular datetime. In the pd.Timestamp case, there are some extra fields that only get set in the __new__ method, leading to a segfault later when they're accessed.

I'm not familiar enough with the C/CPython interface (especially object setup and initialisation) to tell where a fix for this should go, I would assume that the line

cpython/Modules/_datetimemodule.c

Line 5872 in 03e5647

clone = datetime_new(Py_TYPE(self), tuple, NULL);

should be replaced by a call to PyObject_new(Py_TYPE(self), ...), or something similar.

This also affects date.replace and time.replace.