gh-104909: Split some more insts into ops by gvanrossum · Pull Request #109943 · python/cpython

gvanrossum · 2023-09-27T04:01:33Z

I started by adding some code that shows the most deserving non-viable opcodes. Also doubling max trace size again.

I could do this all day, but the stats show there are diminishing returns. Output from python3 Tools/cases_generator/generate_cases.py -v before splitting the top 5:

Read 186 instructions, 46 ops, 207 macros, 12 pseudos, and 14 families from Python/bytecodes.c
The following ops are not viable uops:
1,906,418,185 LOAD_ATTR_METHOD_WITH_VALUES (specialization of LOAD_ATTR)
1,408,928,326 LOAD_ATTR_METHOD_NO_DICT (specialization of LOAD_ATTR)
1,359,753,529 LOAD_ATTR_SLOT (specialization of LOAD_ATTR)
1,064,113,118 STORE_ATTR_SLOT (specialization of STORE_ATTR)
981,677,187 STORE_ATTR_INSTANCE_VALUE (specialization of STORE_ATTR)
958,283,266 YIELD_VALUE 777,829,833 CALL (unspecialized)
490,635,610 SEND_GEN (specialization of SEND)
436,706,438 JUMP_FORWARD 430,588,960 EXTENDED_ARG 391,017,977 JUMP_BACKWARD_NO_INTERRUPT 351,940,012 LOAD_ATTR_MODULE (specialization of LOAD_ATTR)
347,159,058 LOAD_ATTR_WITH_HINT (specialization of LOAD_ATTR)
308,391,927 FOR_ITER (unspecialized)
251,204,370 RETURN_GENERATOR 241,313,761 CALL_LIST_APPEND (specialization of CALL)
177,791,632 CALL_PY_WITH_DEFAULTS (specialization of CALL)
175,204,354 CALL_KW 163,262,604 FOR_ITER_GEN (specialization of FOR_ITER)
146,454,299 BINARY_SUBSCR_GETITEM (specialization of BINARY_SUBSCR)
143,423,491 LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES (specialization of LOAD_ATTR)
136,661,968 STORE_FAST_LOAD_FAST 136,511,856 LOAD_ATTR_CLASS (specialization of LOAD_ATTR)
112,730,831 SEND (unspecialized)
92,715,546 MAKE_CELL 83,410,408 CALL_FUNCTION_EX 69,188,245 CALL_ALLOC_AND_ENTER_INIT (specialization of CALL)
56,944,041 LOAD_ATTR_METHOD_LAZY_DICT (specialization of LOAD_ATTR)
52,280,673 LOAD_ATTR_PROPERTY (specialization of LOAD_ATTR)
50,964,787 STORE_ATTR_WITH_HINT (specialization of STORE_ATTR)
22,352,640 LOAD_ATTR_NONDESCRIPTOR_NO_DICT (specialization of LOAD_ATTR)
8,497,626 RERAISE 6,000,000 END_ASYNC_FOR 5,253,976 BEFORE_WITH 2,897,006 RAISE_VARARGS 2,005,368 IMPORT_FROM 1,968,468 IMPORT_NAME 8,160 BEFORE_ASYNC_WITH 5,907 RESUME (unspecialized)
1,540 LOAD_SUPER_ATTR (unspecialized)
801 CLEANUP_THROW ENTER_EXECUTOR LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN (specialization of LOAD_ATTR)

I took the counts from here (follow the pystats raw link).

Curious what's up with LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN? It was the same on the weekly run a week before; the week before that (Sept 10) the file format was different. @brandtbucher any intuition on this?

Issue: Split opcodes into micro-ops #104909

gvanrossum · 2023-09-27T04:45:16Z

Another couple of questions for @brandtbucher (or @markshannon).

Sometimes there's duplication between two uops, e.g. _GUARD_DORV_VALUES and _STORE_ATTR_INSTANCE_VALUE both set dorv = *_PyObject_DictOrValuesPointer(owner);. In Tier 1 the assumption has always been that the C compiler's optimizer will eliminate the duplication (it's a static inline function). But in the Tier 2 interpreter, we end up executing that code twice (once in each uop), and in the copy-and-patch machine code it also takes up twice the space. It doesn't look too bad, because after removing asserts it's just obj - 3, and a modern CPU should do this very fast, but there might be other occurrences of this kind of duplication that are slightly less minimal.
There's an assert(type_version != 0); in the _GUARD_TYPE_VERSION macro. This existed in most of the instructions I split up, except in LOAD_ATTR_METHOD_NO_DICT. Was that an oversight, or can it be zero for that opcode? (If so, then my PR is incorrect.)
In _LOAD_ATTR_SLOT, which is an "action" uop, I left a DEOPT_IF call. This is unprincipled, but it seems to work, and splitting this off into its own guard would duplicate much of the work this uop does (in particular the indirect load). Thoughts?

Fidget-Spinner · 2023-09-27T10:27:08Z

In _LOAD_ATTR_SLOT, which is an "action" uop, I left a DEOPT_IF call. This is unprincipled, but it seems to work, and splitting this off into its own guard would duplicate much of the work this uop does (in particular the indirect load). Thoughts?

Not Mark or Brandt, but on the optimization side, we would have to treat this as an impure op, which makes most of the optimization opportunities go away. However, we're likely converting some of these to LOAD_CONST anyways so that might mitigate it?

gvanrossum · 2023-09-27T17:10:06Z

Not Mark or Brandt, but on the optimization side, we would have to treat this as an impure op, which makes most of the optimization opportunities go away.

We already have other actions using DEOPT_IF (e.g. _LOAD_GLOBAL_BUILTINS). We can split all these up later. My first goal is just to be able to translate more bytecodes to Tier 2, I'm slowly working my way through the most frequently executed instructions (per the table shown above).

However, we're likely converting some of these to LOAD_CONST anyways so that might mitigate it?

That's still a bit in the future though. And I'm not sure how often LOAD_ATTR_SLOT in particular would become LOAD_CONST (since it specifically fetches an instance attribute). It seems more likely for some other variants like LOAD_ATTR_MODULE, and that would probably load the result from the cache, not be transformed in (some guard +) LOAD_CONST.

gvanrossum · 2023-09-27T17:43:59Z

Going to merge this soon, basically as-is. Perf is same as base (1.00 faster).

gvanrossum · 2023-09-27T17:48:41Z

I can't find a persistent pattern in the failing tests. Maybe it's a timeout in multiprocessing? I believe @vstinner recently made flaky tests more likely to cause CI failure, maybe it's that?

gvanrossum · 2023-09-27T19:11:40Z

Well, okay, some tests fail consistently, so I have to bisect. Sorry for the accusations.

gvanrossum · 2023-09-27T19:36:38Z

Looks like LOAD_ATTR_METHOD_WITH_VALUES is the culprit.

I had dropped one of the DEOPT_IF() calls! :-(

bedevere-bot · 2023-09-27T23:09:15Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable LTO + PGO 3.x has failed when building commit 5bb6f0f.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/29/builds/4838) and take a look at the build logs.
Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/29/builds/4838

Failed tests:

test.test_concurrent_futures.test_shutdown

Failed subtests:

test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolForkserverProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/test/test_concurrent_futures/test_shutdown.py", line 49, in test_interpreter_shutdownself.assertFalse(err)
AssertionError: b'Exception in thread Thread-1:\nTraceback (most recent call last):\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/threading.py", line 1066, in _bootstrap_inner\n self.run()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/concurrent/futures/process.py", line 344, in run\n self.add_call_item_to_queue()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/concurrent/futures/process.py", line 399, in add_call_item_to_queue\n self.call_queue.put(_CallItem(work_id,\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/queues.py", line 94, in put\n self._start_thread()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/queues.py", line 177, in _start_thread\n self._thread.start()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/threading.py", line 985, in start\n _start_new_thread(self._bootstrap, ())\nRuntimeError: can\'t create new thread at interpreter shutdown\nTraceback (most recent call last):\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/forkserver.py", line 274, in main\n code = _serve_one(child_r, fds,\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/forkserver.py", line 313, in _serve_one\n code = spawn._main(child_r, parent_sentinel)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/spawn.py", line 132, in _main\n self = reduction.pickle.load(from_parent)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/synchronize.py", line 115, in __setstate__\n self._semlock = _multiprocessing.SemLock._rebuild(*state)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n' is not false

bedevere-bot · 2023-09-27T23:09:43Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian PGO 3.x has failed when building commit 5bb6f0f.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/249/builds/6464) and take a look at the build logs.
Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/249/builds/6464

Failed tests:

test.test_multiprocessing_fork.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 29, done. remote: Counting objects: 3% (1/29) remote: Counting objects: 6% (2/29) remote: Counting objects: 10% (3/29) remote: Counting objects: 13% (4/29) remote: Counting objects: 17% (5/29) remote: Counting objects: 20% (6/29) remote: Counting objects: 24% (7/29) remote: Counting objects: 27% (8/29) remote: Counting objects: 31% (9/29) remote: Counting objects: 34% (10/29) remote: Counting objects: 37% (11/29) remote: Counting objects: 41% (12/29) remote: Counting objects: 44% (13/29) remote: Counting objects: 48% (14/29) remote: Counting objects: 51% (15/29) remote: Counting objects: 55% (16/29) remote: Counting objects: 58% (17/29) remote: Counting objects: 62% (18/29) remote: Counting objects: 65% (19/29) remote: Counting objects: 68% (20/29) remote: Counting objects: 72% (21/29) remote: Counting objects: 75% (22/29) remote: Counting objects: 79% (23/29) remote: Counting objects: 82% (24/29) remote: Counting objects: 86% (25/29) remote: Counting objects: 89% (26/29) remote: Counting objects: 93% (27/29) remote: Counting objects: 96% (28/29) remote: Counting objects: 100% (29/29) remote: Counting objects: 100% (29/29), done. remote: Compressing objects: 6% (1/15) remote: Compressing objects: 13% (2/15) remote: Compressing objects: 20% (3/15) remote: Compressing objects: 26% (4/15) remote: Compressing objects: 33% (5/15) remote: Compressing objects: 40% (6/15) remote: Compressing objects: 46% (7/15) remote: Compressing objects: 53% (8/15) remote: Compressing objects: 60% (9/15) remote: Compressing objects: 66% (10/15) remote: Compressing objects: 73% (11/15) remote: Compressing objects: 80% (12/15) remote: Compressing objects: 86% (13/15) remote: Compressing objects: 93% (14/15) remote: Compressing objects: 100% (15/15) remote: Compressing objects: 100% (15/15), done. remote: Total 15 (delta 14), reused 1 (delta 0), pack-reused 0 
From https://github.com/python/cpython
* branch main -> FETCH_HEAD
Note: switching to '5bb6f0fcba663e1006f9063d1027ce8bd9f8effb'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 5bb6f0fcba gh-104909: Split some more insts into ops (#109943)
Switched to and reset branch 'main'
find: ‘build’: No such file or directoryfind: ‘build’: No such file or directoryfind: ‘build’: No such file or directoryfind: ‘build’: No such file or directorymake[2]: [Makefile:2821: clean-retain-profile] Error 1 (ignored)
In file included from Python/ceval.c:766:
Python/generated_cases.c.h: In function ‘_PyEval_EvalFrameDefault’:
Python/generated_cases.c.h:2634:31: warning: unused variable ‘tp’ [-Wunused-variable]
2634 | PyTypeObject *tp = Py_TYPE(owner);
|^~
Python/generated_cases.c.h:3593:31: warning: unused variable ‘owner_cls’ [-Wunused-variable]
3593 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~
Python/generated_cases.c.h:3642:31: warning: unused variable ‘owner_cls’ [-Wunused-variable]
3642 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~
In file included from Python/executor.c:87:
Python/executor_cases.c.h: In function ‘_PyUopExecute’:
Python/executor_cases.c.h:1739:27: warning: unused variable ‘tp’ [-Wunused-variable]
1739 | PyTypeObject *tp = Py_TYPE(owner);
|^~
Python/executor_cases.c.h:2302:27: warning: unused variable ‘owner_cls’ [-Wunused-variable]
2302 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~
Python/executor_cases.c.h:2348:27: warning: unused variable ‘owner_cls’ [-Wunused-variable]
2348 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~./Modules/readline.c: In function ‘setup_readline’:
./Modules/readline.c:1256:21: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
1256 | rl_startup_hook = on_startup_hook;
|^
./Modules/readline.c:1258:23: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
1258 | rl_pre_input_hook = on_pre_input_hook;
|^./Modules/socketmodule.c: In function ‘getsockaddrarg’:
./Modules/socketmodule.c:2468:9: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
2468 | strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name));
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/context.c:56: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time
./Modules/_decimal/libmpdec/context.c:56: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time
In file included from Python/ceval.c:766:
Python/generated_cases.c.h: In function ‘_PyEval_EvalFrameDefault’:
Python/generated_cases.c.h:2634:31: warning: unused variable ‘tp’ [-Wunused-variable]
2634 | PyTypeObject *tp = Py_TYPE(owner);
|^~
Python/generated_cases.c.h:3593:31: warning: unused variable ‘owner_cls’ [-Wunused-variable]
3593 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~
Python/generated_cases.c.h:3642:31: warning: unused variable ‘owner_cls’ [-Wunused-variable]
3642 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~
In file included from Python/executor.c:87:
Python/executor_cases.c.h: In function ‘_PyUopExecute’:
Python/executor_cases.c.h:1739:27: warning: unused variable ‘tp’ [-Wunused-variable]
1739 | PyTypeObject *tp = Py_TYPE(owner);
|^~
Python/executor_cases.c.h:2302:27: warning: unused variable ‘owner_cls’ [-Wunused-variable]
2302 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~
Python/executor_cases.c.h:2348:27: warning: unused variable ‘owner_cls’ [-Wunused-variable]
2348 | PyTypeObject *owner_cls = Py_TYPE(owner);
|^~~~~~~~~./Modules/readline.c: In function ‘setup_readline’:
./Modules/readline.c:1256:21: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
1256 | rl_startup_hook = on_startup_hook;
|^
./Modules/readline.c:1258:23: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
1258 | rl_pre_input_hook = on_pre_input_hook;
|^
In function ‘utf8_toUtf8’,
inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
inlined from ‘doParseXmlDecl’ at ./Modules/expat/xmltok.c:1197:13:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
390 | memcpy(*toP, *fromP, bytesToCopy);
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1075:9:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
390 | memcpy(*toP, *fromP, bytesToCopy);
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1081:12:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
390 | memcpy(*toP, *fromP, bytesToCopy);
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1088:9:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
390 | memcpy(*toP, *fromP, bytesToCopy);
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
In function ‘utf8_toUtf8’,
inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1115:7:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
390 | memcpy(*toP, *fromP, bytesToCopy);
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1128:9:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
390 | memcpy(*toP, *fromP, bytesToCopy);
|^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~make: *** [Makefile:2040: buildbottest] Error 5

bedevere-bot · 2023-09-27T23:36:55Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 LTO 3.x has failed when building commit 5bb6f0f.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/567/builds/4991) and take a look at the build logs.
Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/567/builds/4991

Failed tests:

test.test_multiprocessing_fork.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs

Note: switching to '5bb6f0fcba663e1006f9063d1027ce8bd9f8effb'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 5bb6f0fcba gh-104909: Split some more insts into ops (#109943)
Switched to and reset branch 'main'
In file included from Python/ceval.c:766:
Python/generated_cases.c.h: In function ‘_PyEval_EvalFrameDefault’:
Python/generated_cases.c.h:2634:31: warning: unused variable ‘tp’ [-Wunused-variable]
PyTypeObject *tp = Py_TYPE(owner);^~
Python/generated_cases.c.h:3593:31: warning: unused variable ‘owner_cls’ [-Wunused-variable]
PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~
Python/generated_cases.c.h:3642:31: warning: unused variable ‘owner_cls’ [-Wunused-variable]
PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~
In file included from Python/executor.c:87:
Python/executor_cases.c.h: In function ‘_PyUopExecute’:
Python/executor_cases.c.h:1739:27: warning: unused variable ‘tp’ [-Wunused-variable]
PyTypeObject *tp = Py_TYPE(owner);^~
Python/executor_cases.c.h:2302:27: warning: unused variable ‘owner_cls’ [-Wunused-variable]
PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~
Python/executor_cases.c.h:2348:27: warning: unused variable ‘owner_cls’ [-Wunused-variable]
PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~
./Include/internal/pycore_semaphore.h:54:1: warning: type of ‘_PySemaphore_Wakeup’ does not match original declaration [-Wlto-type-mismatch]
_PySemaphore_Wakeup(_PySemaphore *sema);
^
Python/parking_lot.c:197:1: note: ‘_PySemaphore_Wakeup’ was previously declared here
_PySemaphore_Wakeup(_PySemaphore *sema)
^
Python/parking_lot.c:197:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:57:18: warning: type of ‘_PySemaphore_Init’ does not match original declaration [-Wlto-type-mismatch]
PyAPI_FUNC(void) _PySemaphore_Init(_PySemaphore *sema);
^
Python/parking_lot.c:53:1: note: ‘_PySemaphore_Init’ was previously declared here
_PySemaphore_Init(_PySemaphore *sema)
^
Python/parking_lot.c:53:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:49:1: warning: type of ‘_PySemaphore_Wait’ does not match original declaration [-Wlto-type-mismatch]
_PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout_ns, int detach);
^
Python/parking_lot.c:178:1: note: ‘_PySemaphore_Wait’ was previously declared here
_PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout, int detach)
^
Python/parking_lot.c:178:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:58:18: warning: type of ‘_PySemaphore_Destroy’ does not match original declaration [-Wlto-type-mismatch]
PyAPI_FUNC(void) _PySemaphore_Destroy(_PySemaphore *sema);
^
Python/parking_lot.c:81:1: note: ‘_PySemaphore_Destroy’ was previously declared here
_PySemaphore_Destroy(_PySemaphore *sema)
^
Python/parking_lot.c:81:1: note: code may be misoptimized unless -fno-strict-aliasing is used
make: *** [Makefile:2040: buildbottest] Error 5

bedevere-bot · 2023-09-28T00:01:15Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM Raspbian 3.x has failed when building commit 5bb6f0f.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/424/builds/5033) and take a look at the build logs.
Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/424/builds/5033

Failed tests:

test.test_multiprocessing_spawn.test_misc

Failed subtests:

test_dict - test.test_multiprocessing_spawn.test_misc.TestSyncManagerTypes.test_dict
test_bounded_semaphore - test.test_multiprocessing_spawn.test_misc.TestSyncManagerTypes.test_bounded_semaphore
test_condition - test.test_multiprocessing_spawn.test_misc.TestSyncManagerTypes.test_condition

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5931, in test_dictself.run_worker(self._test_dict, o)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5799, in run_workerself.wait_proc_exit()
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5781, in wait_proc_exit
join_process(self.proc)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 100, in join_process
threading_helper.join_thread(process)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/threading_helper.py", line 113, in join_threadraiseAssertionError(msg)
AssertionError: failed to join the thread in 52.5 seconds
Traceback (most recent call last):
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5854, in test_conditionself.run_worker(self._test_condition, o)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5799, in run_workerself.wait_proc_exit()
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5781, in wait_proc_exit
join_process(self.proc)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 100, in join_process
threading_helper.join_thread(process)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/threading_helper.py", line 113, in join_threadraiseAssertionError(msg)
AssertionError: failed to join the thread in 52.5 seconds
Traceback (most recent call last):
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5845, in test_bounded_semaphoreself.test_semaphore(sname="BoundedSemaphore")
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5841, in test_semaphoreself.run_worker(self._test_semaphore, o)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5799, in run_workerself.wait_proc_exit()
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5781, in wait_proc_exit
join_process(self.proc)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 100, in join_process
threading_helper.join_thread(process)
File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/threading_helper.py", line 113, in join_threadraiseAssertionError(msg)
AssertionError: failed to join the thread in 52.5 seconds

These are the most popular specializations of `LOAD_ATTR` and `STORE_ATTR` that weren't already viable uops: * Split LOAD_ATTR_METHOD_WITH_VALUES * Split LOAD_ATTR_METHOD_NO_DICT * Split LOAD_ATTR_SLOT * Split STORE_ATTR_SLOT * Split STORE_ATTR_INSTANCE_VALUE Also: * Add `-v` flag to code generator which prints a list of non-viable uops (easter-egg: it can print execution counts -- see source) * Double _Py_UOP_MAX_TRACE_LENGTH to 128 I had dropped one of the DEOPT_IF() calls! :-(

vstinner · 2023-09-28T11:46:38Z

I believe @vstinner recently made flaky tests more likely to cause CI failure, maybe it's that?

Right. Tests are now run with --rerun-fail on GitHub Actions and buildbots: any test marks the whole build as a failure.

These are the most popular specializations of `LOAD_ATTR` and `STORE_ATTR` that weren't already viable uops: * Split LOAD_ATTR_METHOD_WITH_VALUES * Split LOAD_ATTR_METHOD_NO_DICT * Split LOAD_ATTR_SLOT * Split STORE_ATTR_SLOT * Split STORE_ATTR_INSTANCE_VALUE Also: * Add `-v` flag to code generator which prints a list of non-viable uops (easter-egg: it can print execution counts -- see source) * Double _Py_UOP_MAX_TRACE_LENGTH to 128 I had dropped one of the DEOPT_IF() calls! :-(

gvanrossum added 8 commits September 26, 2023 17:43

Print non-viable uops
2ca26bb

Sort non-viable opcodes by execution count
e19517c

Double _Py_UOP_MAX_TRACE_LENGTH to 128
8d1a856

Split LOAD_ATTR_METHOD_WITH_VALUES
3796e89

Split LOAD_ATTR_METHOD_NO_DICT
6d816ed

Split LOAD_ATTR_SLOT (weird: action may deopt)
5ba03d1

Split STORE_ATTR_SLOT
5d0fa92

Split STORE_ATTR_INSTANCE_VALUE
b54eee3

gvanrossum force-pushed the load-attr-uops branch from 7bdddb0 to b54eee3Compare September 27, 2023 04:27

gvanrossum added the skip news label Sep 27, 2023

gvanrossum changed the title ~~Split some more insts into ops~~gh-104909: Split some more insts into opsSep 27, 2023

bedevere-appBot mentioned this pull request Sep 27, 2023
Split opcodes into micro-ops #104909
Closed

markshannon reviewed Sep 27, 2023
View reviewed changes

Comment threadTools/cases_generator/analysis.py

markshannon reviewed Sep 27, 2023
View reviewed changes

Comment threadPython/bytecodes.c

gvanrossum marked this pull request as ready for review September 27, 2023 17:44

bedevere-appBot added the awaiting review label Sep 27, 2023

Merge branch 'main' into load-attr-uops
24b4ec9

Fix LOAD_ATTR_METHOD_WITH_VALUES
85fb1e7
I had dropped one of the DEOPT_IF() calls! :-(

gvanrossum merged commit 5bb6f0f into python:mainSep 27, 2023

bedevere-appBot removed the awaiting review label Sep 27, 2023

gvanrossum deleted the load-attr-uops branch September 27, 2023 22:27

gvanrossum mentioned this pull request Sep 28, 2023
How to handle YIELD_VALUE in Tier 2 faster-cpython/ideas#628
Open

This was referenced Sep 28, 2023
Warning: unused variable ‘owner_cls’ in generated_cases.c.h#110020
Closed
gh-110020: Fix unused variable warnings in bytecodes.c#110023
Merged

Uh oh!

Uh oh!

Conversation

gvanrossum commented Sep 27, 2023• edited by bedevere-app BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum commented Sep 27, 2023

Uh oh!

Fidget-Spinner commented Sep 27, 2023

Uh oh!

Uh oh!

Uh oh!

gvanrossum commented Sep 27, 2023

Uh oh!

gvanrossum commented Sep 27, 2023

Uh oh!

gvanrossum commented Sep 27, 2023

Uh oh!

gvanrossum commented Sep 27, 2023

Uh oh!

gvanrossum commented Sep 27, 2023

Uh oh!

bedevere-bot commented Sep 27, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 27, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 27, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 28, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

vstinner commented Sep 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gvanrossum commented Sep 27, 2023•
edited by bedevere-app Bot
Loading