gh-91052: Add C API for watching dictionaries by carljm · Pull Request #31787 · python/cpython

carljm · 2022-03-09T14:31:46Z

Notes on the PR for reviewers:

Since this API is intended for use by extensions, and _Py* APIs are internal-only, I added the API as public (Py*). But I think it should perhaps be CPython-specific, so I put it in Include/cpython/dictobject.h, not Include/dictobject.h. Not sure about any of this, though, so happy to change as requested.

Issue: add support for watching writes to selected dictionaries #91052

MaxwellDupre · 2022-05-02T10:36:41Z

Do you have a test harness/or something I can use (without too much effort :)? I could just run the Python test suite but that may be missing the point.

carljm · 2022-05-02T15:55:34Z

Do you have a test harness/or something I can use (without too much effort :)? I could just run the Python test suite but that may be missing the point.

Hi! I'm not sure what you are trying to achieve -- can you clarify your goals and how they relate to this PR?

ghost · 2022-10-03T22:36:37Z

All commit authors signed the Contributor License Agreement.

carljm · 2022-10-04T00:07:30Z

@markshannon I've updated this PR to the design we discussed, would be glad for your review. I'll try to get some updated pyperformance numbers tomorrow.

carljm · 2022-10-04T21:37:28Z

Shows neutral PyPerformance results: https://gist.github.com/carljm/fa125fe5147146cd62ae10485b3fc31b

markshannon

Generally looks good. A few comments.

* main: (66 commits) pythongh-65961: Raise `DeprecationWarning` when `__package__` differs from `__spec__.parent` (python#97879) docs(typing): add "see PEP 675" to LiteralString (python#97926) pythongh-97850: Remove all known instances of module_repr() (python#97876) I changed my surname early this year (python#96671) pythongh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) (python#97768) pythongh-91539: improve performance of get_proxies_environment (python#91566) build(deps): bump actions/stale from 5 to 6 (python#97701) pythonGH-95172 Make the same version `versionadded` oneline (python#95172) pythongh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (python#32073) pythongh-93738: Documentation C syntax (Function glob patterns -> literal markup) (python#97774) pythongh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (python#97896) pythongh-95196: Disable incorrect pickling of the C implemented classmethod descriptors (pythonGH-96383) pythongh-97758: Fix a crash in getpath_joinpath() called without arguments (pythonGH-97759) pythongh-74696: Pass root_dir to custom archivers which support it (pythonGH-94251) pythongh-97661: Improve accuracy of sqlite3.Cursor.fetchone docs (python#97662) pythongh-87092: bring compiler code closer to a preprocessing-opt-assembler organisation (pythonGH-97644) pythonGH-96704: Add {Task,Handle}.get_context(), use it in call_exception_handler() (python#96756) pythongh-93738: Documentation C syntax (:c:type:`PyTypeObject*` -> :c:expr:`PyTypeObject*`) (python#97778) pythongh-97825: fix AttributeError when calling subprocess.check_output(input=None) with encoding or errors args (python#97826) Add re.VERBOSE flag documentation example (python#97678) ...

* main: pythonGH-88050: fix race in closing subprocess pipe in asyncio (python#97951) pythongh-93738: Disallow pre-v3 syntax in the C domain (python#97962) pythongh-95986: Fix the example using match keyword (python#95989) pythongh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (pythonGH-97944) pythongh-94808: Cover `PyUnicode_Count` in CAPI (python#96929) pythongh-94808: Cover `PyObject_PyBytes` case with custom `__bytes__` method (python#96610) pythongh-95691: Doc BufferedWriter and BufferedReader (python#95703) pythonGH-88968: Add notes about socket ownership transfers (python#97936) pythongh-96865: [Enum] fix Flag to use CONFORM boundary (pythonGH-97528)

markshannon

A few more comments, but looking good.

* main: fixespythongh-96078: os.sched_yield release the GIL while calling sched_yield(2). (pythongh-97965) pythongh-65961: Do not rely solely on `__cached__` (pythonGH-97990) pythongh-97850: Remove the open issues section from the import reference (python#97935) Docs: pin sphinx-lint (pythonGH-97992) pythongh-94590: add signatures to operator itemgetter, attrgetter, methodcaller (python#94591) Add Pynche's move to the What's new in 3.11 (python#97974) pythongh-97781: Apply changes from importlib_metadata 5. (pythonGH-97785) pythongh-86482: Document assignment expression need for ()s (python#23291) pythongh-97943: PyFunction_GetAnnotations should return a borrowed reference. (python#97949) pythongh-94808: Coverage: Test that maximum indentation level is handled (python#95926)

markshannon · 2022-10-07T00:07:16Z

LGTM, thanks.

sweeneyde · 2022-10-07T05:39:50Z

I think this may have introduced some refleaks into test_capi.

carljm · 2022-10-07T05:42:36Z

Thanks for the heads up, I'll take a look.

iritkatriel · 2022-10-07T12:08:30Z

+ self.assertIs(unraisable.object, d)
+ self.assertEqual(str(unraisable.exc_value), "boom!")
+
+ def test_two_watchers(self):


This test is ref leaking on my PR: #98001

Ah probably fixed in #98017 ?

vstinner · 2022-10-07T15:23:48Z

For the C API documentation changes, you should add .. versionadded:: 3.12 to new functions, and document them in Doc/whatsnew/3.12.rst:
https://docs.python.org/dev/whatsnew/3.12.html#id2

carljm · 2022-10-07T19:34:46Z

Thanks @vstinner , I included these changes in #98055.

* main: bpo-35540 dataclasses.asdict now supports defaultdict fields (pythongh-32056) pythonGH-91052: Add C API for watching dictionaries (pythonGH-31787) bpo-38693: Use f-strings instead of str.format() within importlib (python#17058)

carljm requested review from markshannon and methane as code owners March 9, 2022 14:31

bedevere-bot added the awaiting review label Mar 9, 2022

the-knights-who-say-ni added the CLA signed label Mar 9, 2022

itamaro reviewed Mar 9, 2022
View reviewed changes

Comment threadObjects/dictobject.c Outdated
Comment threadObjects/dictobject.c Outdated

carljm force-pushed the dictwatch branch 3 times, most recently from eed27b4 to d22463bCompare March 14, 2022 23:14

carljm mentioned this pull request May 3, 2022
add support for watching writes to selected dictionaries #91052
Closed

ezio-melotti removed the CLA signed label Jul 13, 2022

carljm force-pushed the dictwatch branch from 4623234 to 11caa9eCompare October 3, 2022 23:06

pythongh-91052: Add C API for watching dictionaries
0447232

carljm force-pushed the dictwatch branch from 11caa9e to 0447232Compare October 3, 2022 23:10

carljm changed the title ~~bpo-46896: Add C API for watching dictionaries~~gh-91052: Add C API for watching dictionariesOct 3, 2022

carljm added 2 commits October 4, 2022 09:51

Add second-watcher tests, fix MSVC warning
1c7f814

Over-engineer the watcher bits check
6e5cee2

Fidget-Spinner reviewed Oct 5, 2022
View reviewed changes

Comment threadModules/_testcapimodule.c Outdated

markshannon reviewed Oct 5, 2022
View reviewed changes

Comment threadInclude/internal/pycore_dict.h Outdated
Comment threadInclude/internal/pycore_interp.h Outdated
Comment threadModules/_testcapimodule.c Outdated
Comment threadModules/_testcapimodule.c Outdated
Comment threadModules/_testcapimodule.c

carljm added 4 commits October 5, 2022 15:04

Address review comments
7dcf9b0

Minor cleanups
fb19617

carljm mentioned this pull request Oct 6, 2022
gh-91051: allow setting a callback hook on PyType_Modified #97875
Merged

markshannon reviewed Oct 6, 2022
View reviewed changes

Comment threadDoc/c-api/dict.rst
Comment threadDoc/c-api/dict.rst Outdated
Comment threadDoc/c-api/dict.rst Outdated
Comment threadInclude/cpython/dictobject.h Outdated

Address further review comments
cc1d0b7

markshannon merged commit a4b7794 into python:mainOct 7, 2022

bedevere-bot removed the awaiting review label Oct 7, 2022

carljm deleted the dictwatch branch October 7, 2022 00:14

carljm mentioned this pull request Oct 7, 2022
Fix memory leaks in test_capi #98017
Merged

iritkatriel reviewed Oct 7, 2022
View reviewed changes

carljm mentioned this pull request Oct 7, 2022
gh-91052: Add PyDict_Unwatch for unwatching a dictionary #98055
Merged

mpage pushed a commit to mpage/cpython that referenced this pull request Oct 11, 2022
pythonGH-91052: Add C API for watching dictionaries (pythonGH-31787)
4f9e610

erlend-aasland mentioned this pull request Nov 10, 2022
Split up _testcapimodule.c #93649
Open

Uh oh!

Uh oh!

Conversation

carljm commented Mar 9, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MaxwellDupre commented May 2, 2022

Uh oh!

carljm commented May 2, 2022

Uh oh!

ghost commented Oct 3, 2022• edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

carljm commented Oct 4, 2022

Uh oh!

carljm commented Oct 4, 2022

Uh oh!

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markshannon commented Oct 7, 2022

Uh oh!

sweeneyde commented Oct 7, 2022

Uh oh!

carljm commented Oct 7, 2022

Uh oh!

iritkatrielOct 7, 2022

Choose a reason for hiding this comment

Uh oh!

iritkatrielOct 7, 2022

Choose a reason for hiding this comment

Uh oh!

vstinner commented Oct 7, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

carljm commented Oct 7, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

carljm commented Mar 9, 2022•
edited
Loading

ghost commented Oct 3, 2022•
edited by ghost
Loading

vstinner commented Oct 7, 2022•
edited
Loading