bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3

[erlend-aasland]

https://bugs.python.org/issue46541

[erlend-aasland]

Assuming we skip news, since this is an internal change only.

[ericsnowcurrently]

The main reason I didn't do this earlier is because our (non-builtin) extension modules probably shouldn't be relying on internal API. Instead we should try to use only public API (and sometimes "private", AKA public API with a leading underscore).

How bad would it be to replace uses of _Py_IDENTIFIER() with PyUnicode_InternFromString()?

[erlend-aasland]

The main reason I didn't do this earlier is because our (non-builtin) extension modules probably shouldn't be relying on internal API.

It's fully possible to build any stdlib extension as a built-in, even sqlite3.

How bad would it be to replace uses of _Py_IDENTIFIER() with PyUnicode_InternFromString()?

Most users will call execute and friend on a cursor object, not on a connection object, so they won't notice neither performance improvements nor regressions. I'm not sure about the adapter case though; I don't know how widely used they actually are.

[erlend-aasland]

The main reason I didn't do this earlier is because our (non-builtin) extension modules probably shouldn't be relying on internal API. Instead we should try to use only public API (and sometimes "private", AKA public API with a leading underscore).

OTOH, I'm totally fine with mainly using public APIs in the sqlite3 module. I'll rewrite it to use public APIs instead.

[ericsnowcurrently]

BTW, thanks for working on this!

[erlend-aasland]

BTW, thanks for working on this!

Likewise :)

[erlend-aasland]

PTAL. This should result in a small speedup compared to main.

The sqlite3.Connection.cursor() calls are now made directly. Ideally we should do the same with the Cursor.execute* calls, but sqlite3 is inconveniently implemented across multiple files¹, so we can't just call the cursor impl methods directly. I changed the scope of _pysqlite_query_execute, so we can at least speed up the Connection.execute() and Connection.executemany() calls. The remaining methods now use interned strings, with the module always keeping a strong ref to __adapt__, __conform__, and upper to speed up adapters/converters, and finalize to speed up user-defined functions. I did the same for executemany; it is probably not used in hot code, but following the same pattern simplified the code a bit.

Footnotes

1. Consolidating all the Module/_sqlite/*.[ch] files into Module/_sqlite3.[ch] would be very helpful. ↩

[corona10]

lgtm

[erlend-aasland]

Thanks for your review and thoughts, @corona10 & @ericsnowcurrently