Improve guidelines for GC protocol implementation for heap types

[picnixz]

This is a follow-up to #125962 which added some guidelines:

• Use type->tp_alloc instead of PyObject_New and PyObject_GC_New.
• Use type->tp_free instead of PyObject_Free and PyObject_GC_Del.

Those two recommendations were introduced to facilitate adding the Py_TPFLAGS_HAVE_GC flag to a heap type (those types must (should?) implement the GC protocol, at least according to the docs: https://docs.python.org/3/c-api/gcsupport.html#supporting-cycle-detection).

Now, the docs should indicate that:

• type->tp_alloc may call PyObject_GC_Track, depending on Py_TPFLAGS_HAVE_GC. Some users could be surprised by this (like me in gh-116946: fully implement GC protocol for bz2 objects #138266).
• https://docs.python.org/3.14/c-api/gcsupport.html#supporting-cycle-detection should be updated because it mentions using PyObject_GC_New, which itself mentions using tp_alloc directly. Better to just mention tp_alloc.

What I am actually worried about is:

Constructors for container types must conform to two rules:

• The memory for the object must be allocated using PyObject_GC_New or PyObject_GC_NewVar.
• Once all the fields which may contain references to other containers are initialized, it must call PyObject_GC_Track().

Now, tp_alloc automatically calls PyObject_GC_Track so users won't be able to pre-initialize fields, so I suggest that we mention this.

Outdated discussion

If people need to first initialize fields, maybe we should recommend constructing them first:

static PyObject *
object_new(PyTypeObject *type)
{
 T *self = NULL;
 PyObject *f1, *f2, *f3;
 
 f1 = do1();
 if (f1 == NULL) { goto error_pre_init; }
 f2 = do2();
 if (f2 == NULL) { goto error_pre_init; }
 f3 = do3();
 if (f3 == NULL) { goto error_pre_init; }
 
 self = (T *)type->tp_alloc(type, 0);
 if (self == NULL) {
 goto error_pre_init;
 }
 self->f1 = f1;
 self->f2 = f2;
 self->f3 = f3;
 f1 = f2 = f3 = NULL;

 if (finalize(self) < 0) {
 goto error;
 }
 return (PyObject *)self;

error_pre_init:
 Py_XDECREF(f1);
 Py_XDECREF(f2);
 Py_XDECREF(f3);
 return NULL;

error_post_init:
 Py_DECREF(self);
 return NULL;
}

cc @ZeroIntensity

[ZeroIntensity]

Now, tp_alloc automatically calls PyObject_GC_Track so users won't be able to pre-initialize fields. If people need to first initialize fields, maybe we should recommend constructing them first

I don't think this will actually be an issue in practice. tp_alloc will zero the memory, so tp_clear and tp_traverse can be safely called. If the constructor calls into Python code, the object shouldn't be used anyway, per the note in gc.get_referrers:

Care must be taken when using objects returned by get_referrers() because some of them could still be under construction and hence in a temporarily invalid state. Avoid using get_referrers() for any purpose other than debugging.

[picnixz]

Yes but what if someone is trying to debug stuff? that would be possibly annoying for this specific task! so I still think we should either warn them about this or we shouldn't say "don't track until the fields are initialized" since we don't even follow this!

[ZeroIntensity]

don't track until the fields are initialized

We do follow that -- the fields are initialized to NULL, rather than an object reference. We could warn about it, though.

[picnixz]

In light of #116946 (comment) and #138329 (comment), immutable empty heap types do not need to implement the GC protocol because they will never contribute to a cycle. This may be documented as well.