Possible double Py_XDECREF in cpython typeobject.c

[Wesley-Jzy]

BPO	44588
Nosy	@encukou, @iritkatriel, @Wesley-Jzy
PRs	bpo-42747: Remove Py_TPFLAGS_HAVE_AM_SEND and make Py_TPFLAGS_HAVE_VERSION_TAG no-op #27260

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-07-09.07:42:56.699>
labels = ['expert-C-API', 'type-crash', '3.11']
title = 'Possible double Py_XDECREF in cpython typeobject.c'
updated_at = <Date 2021-07-21.02:41:27.600>
user = 'https://github.com/Wesley-Jzy'

bugs.python.org fields:

activity = <Date 2021-07-21.02:41:27.600>
actor = 'Wesley-Jzy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2021-07-09.07:42:56.699>
creator = 'Wesley-Jzy'
dependencies = []
files = []
hgrepos = []
issue_num = 44588
keywords = ['patch']
message_count = 8.0
messages = ['397188', '397189', '397190', '397191', '397192', '397193', '397875', '397921']
nosy_count = 3.0
nosy_names = ['petr.viktorin', 'iritkatriel', 'Wesley-Jzy']
pr_nums = ['27260']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue44588'
versions = ['Python 3.11']

[Wesley-Jzy]

The type_mro_modified() function in Object/typeobject.c may produce double Py_XDECREF on mro_meth and type_mro_meth when enter the code:
if (!_PyType_HasFeature(cls, Py_TPFLAGS_HAVE_VERSION_TAG) ||
!PyType_IsSubtype(type, cls)) {
goto clear;
}

I think
mro_meth = NULL;
type_mro_meth = NULL;
should be added after the first time Py_XDECREF them.

[Wesley-Jzy]

Maybe a little complicared but you can still construct a case to trigger the double free action, thus causing a SIGABRT.

Program received signal SIGABRT, Aborted.
0x00007ffff7c3718b in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff7c3718b in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff7c16859 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x000000000051e94e in fatal_error (prefix=prefix@entry=0x0, msg=msg@entry=0x6a38e4 "_PyObject_AssertFailed", status=status@entry=-1) at ../Python/pylifecycle.c:2183
#3 0x00000000005213b0 in Py_FatalError (msg=msg@entry=0x6a38e4 "_PyObject_AssertFailed") at ../Python/pylifecycle.c:2193
#4 0x0000000000470dd9 in _PyObject_AssertFailed (obj=obj@entry=0x7ffff789c350, expr=expr@entry=0x0, msg=msg@entry=0x6a392d "object has negative ref count", file=file@entry=0x691111 "../Include/object.h", line=line@entry=541,
function=function@entry=0x6a4a20 <func.15840> "_Py_NegativeRefcount") at ../Objects/object.c:2200
#5 0x0000000000470ee1 in _Py_NegativeRefcount (filename=filename@entry=0x691111 "../Include/object.h", lineno=lineno@entry=541, op=op@entry=0x7ffff789c350) at ../Objects/object.c:235
#6 0x0000000000490ad8 in _Py_DECREF (op=0x7ffff789c350, lineno=541, filename=0x691111 "../Include/object.h") at ../Include/object.h:473
#7 _Py_XDECREF (op=0x7ffff789c350) at ../Include/object.h:541
#8 type_mro_modified (type=type@entry=0xb17400, bases=bases@entry=0x7ffff5917890) at ../Objects/typeobject.c:343
#9 0x00000000004940ca in mro_internal (type=type@entry=0xb17400, p_old_mro=p_old_mro@entry=0x0) at ../Objects/typeobject.c:1961
#10 0x000000000048c816 in PyType_Ready (type=type@entry=0xb17400) at ../Objects/typeobject.c:5345
#11 0x0000000000493974 in type_new (metatype=<optimized out>, args=<optimized out>, kwds=0x0) at ../Objects/typeobject.c:2806

[iritkatriel]

Do you have a small piece of code reproducing it that you can upload?

[Wesley-Jzy]

I have no detailed code. The way to I produce it is that using PyType_FromSpec() to generate a type A without the flag Py_TPFLAGS_DEFAULT(which sets the flag Py_TPFLAGS_HAVE_VERSION_TAG).

Then compile and run in Python.

from my_pkg import A
class Time1(Time):
 def __init__(self):
 pass

import multiprocessing (This import is a method to 100% reproduce it. Another way is to use python3-dbg running the code.)

[Wesley-Jzy]

Sorry, not inherited from Time, Time is the class I use in a real project.
Time -> A

[Wesley-Jzy]

I just take over a Python3.6 project from a friend, migrating it to the newest Python version. Then this problem happened. After debugging, I think it's a possible double Py_XDECREF if using C-API like this.
But I'm not familiar with Python C-API before so I'm not sure whether it's a problem or just a misuse about tp_flags.

[encukou]

Please use Py_TPFLAGS_DEFAULT when creating objects. Do you have a reason to not do it?

The flag Py_TPFLAGS_HAVE_VERSION_TAG is unneeded and Python should not check for it. There's bpo-42747 open for that already; I sent a PR for it.

[Wesley-Jzy]

Thanks for replying.
I did fix my code by adding Py_TPFLAGS_DEFAULT. It's okay. I just think the behavior is a little strange when I don't set the default flag, thus adding this issue.