Uh oh!
There was an error while loading. Please reload this page.
GH-91079: Decouple C stack overflow checks from Python recursion checks. - #96510
Conversation
bedevere-bot
commented
Sep 5, 2022
🤖 New build scheduled with the buildbot fleet by @markshannon for commit e8a1bed 🤖 If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again. |
bedevere-bot
commented
Sep 5, 2022
🤖 New build scheduled with the buildbot fleet by @markshannon for commit 7e21a74 🤖 If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again. |
bedevere-bot
commented
Sep 6, 2022
🤖 New build scheduled with the buildbot fleet by @markshannon for commit 65cca7d 🤖 If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again. |
bedevere-bot
commented
Sep 13, 2022
🤖 New build scheduled with the buildbot fleet by @markshannon for commit d75797c 🤖 If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again. |
| # 1,000 on most systems | ||
| limit = sys.getrecursionlimit() | ||
| code = "lambda: " + "+".join(f"_number_{i}" for i in range(limit)) | ||
| # Need more than 256 variables to use EXTENDED_ARGS |
There was a problem hiding this comment.
I assume EXTENDED_ARGS has stack implications? explaining the "why" of this here would be useful.
| # and is equal to recursion_limit when _gen_throw() calls | ||
| # PyErr_NormalizeException(). | ||
| recurse(setrecursionlimit(depth + 2) - depth) | ||
| recurse(5000) |
There was a problem hiding this comment.
There's a constant of 5000 used in all sorts of tests for the same purpose of being "too high" for recursion with this PR. I suggest making this a named test.support constant and referring to it instead of mystery constants spread throughout the test suite.
There was a problem hiding this comment.
(and where other constants are derived from that to be higher or lower scale, use math from the main value?)
| /* WASI has limited call stack. Python's recursion limit depends on code | ||
| layout, optimization, and WASI runtime. Wasmtime can handle about 700 | ||
| recursions, sometimes less. 500 is a more conservative limit. */ | ||
| #ifndef Py_DEFAULT_RECURSION_LIMIT |
| # define Py_DEFAULT_RECURSION_LIMIT 1000 | ||
| # endif | ||
| #endif | ||
| #define Py_DEFAULT_RECURSION_LIMIT 1000 |
There was a problem hiding this comment.
keep the ifndef around this, those exist to allow someone setting their own via CFLAGS.
63f55aa to
d1abed5Compare
gpshead
left a comment
There was a problem hiding this comment.
this tied in nicely with the 3.11 talk :)
tacaswell
commented
Oct 7, 2022
This broke compilation of https://github.com/python-greenlet/greenlet/ Doing the renames as suggested by the compiler fixes it (although from reading the PR here, maybe Is this an unintended side effect of this change and or does greenlets need to adapt? |
gpshead
commented
Oct 7, 2022
Not being familiar with greenlet much I suspect it wants |
gpshead
commented
Oct 7, 2022
BTW @tacaswell major kudos to you for testing against CPython |
Thanks, I'll get a PR to greenlet done in the next few days (I am also barely familiar with greenlet but it is a dependency of a dependency of something I care about). I have built a whole rube-goldberg machine to test CPython |
dimpase
commented
Nov 17, 2023
E.g. the very basic, really CS101, recursion speedup with cache # fib.py importsyssys.setrecursionlimit(2000)
fromfunctoolsimportcache@cachedeffib(n):
ifn<1: return0ifn==1: return1returnfib(n-1) +fib(n-2)
print(fib(500))got broken by this PR. Perhaps needless to say, on the latest main branch, --- a/Include/cpython/pystate.h+++ b/Include/cpython/pystate.h@@ -225,7 +225,7 @@ struct _ts {
# define Py_C_RECURSION_LIMIT 500
#else
// This value is duplicated in Lib/test/support/__init__.py
-# define Py_C_RECURSION_LIMIT 1500+# define Py_C_RECURSION_LIMIT 3000
#endifmakes it work (I didn't try to find the minimal needed increase for |
gpshead
commented
Nov 18, 2023
Please stop using this long merged and closed PR as a forum, nobody liatens here. This is not an issue tracker. If you believe there is a bug, file a new issue. |
dimpase
commented
Nov 20, 2023
These are filed. I also don't understand why this was merged, while not conforming to python/steering-council#102 (they asked for |
On Python 3.12, this provokes a stack overflow in the scheduler. It is not quite clear why that's the case; pure-Python recursion even with generators seems to respond well to setrecursionlimit(): ```py def f(n): if n: yield from f(n-1) else: yield 5 import sys sys.setrecursionlimit(3500) print(list(f(3400))) ``` That said, there have been [behavior](python/cpython#96510) [changes](python/cpython#112215) in Py3.12 in this regard, but it is not clear what exactly about Loopy's behavior makes it fall into the 'bad' case.
On Python 3.12, this provokes a stack overflow in the scheduler. It is not quite clear why that's the case; pure-Python recursion even with generators seems to respond well to setrecursionlimit(): ```py def f(n): if n: yield from f(n-1) else: yield 5 import sys sys.setrecursionlimit(3500) print(list(f(3400))) ``` That said, there have been [behavior](python/cpython#96510) [changes](python/cpython#112215) in Py3.12 in this regard, but it is not clear what exactly about Loopy's behavior makes it fall into the 'bad' case.
On Python 3.12, this provokes a stack overflow in the scheduler. It is not quite clear why that's the case; pure-Python recursion even with generators seems to respond well to setrecursionlimit(): ```py def f(n): if n: yield from f(n-1) else: yield 5 import sys sys.setrecursionlimit(3500) print(list(f(3400))) ``` That said, there have been [behavior](python/cpython#96510) [changes](python/cpython#112215) in Py3.12 in this regard, but it is not clear what exactly about Loopy's behavior makes it fall into the 'bad' case.
Uh oh!
There was an error while loading. Please reload this page.