gh-90095: Make .pdbrc work properly and add some reasonable tests

[gaogaotiantian]

pdb -c and .pdbrc works in a very weird way now. cmd.Cmd has cmdqueue which is perfect for loading user defined commands yet we decided to make our own path to hack it. This makes the special commands like commands or ;; fail because they rely on some cmd.Cmd mechanics.

This PR utilizes cmdqueue (and fixed where it was used wrong) which fixed the problem mentioned in #90095. With this implementation, the commands in .pdbrc and passed in with -c are treated almost exactly the same as the commands the users type in.

Two existing tests related to .pdbrc are removed as ... they were not testing they thought they were.

• test_read_pdbrc_with_ascii_encoding simply does not make sense - it basically tests that system with ascii encoding can't print unicode - it has almost nothing to do with pdb. It has nothing to do with enforcing .pdbrc to be utf-8.
• test_readrc_kwarg passes only because the commands in .pdbrc does not execute - I don't understand what's the point of the test.

Reasonable tests are added with the PR to test .pdbrc. The testing framework is improved a bit to make writing .pdbrc tests easier.

• Issue: Pdb can't use the commands through -c or .pdbrc files #90095

[Delengowski]

I'd really like to echo that this become included. I myself wanted to take a crack this and am excited to see a PR open for it. Defining commands in .pdbrc would be a great use.

[gaogaotiantian]

Do we consider this as a bug @iritkatriel ? (backport?)

[iritkatriel]

Could backporting break working code?

[Delengowski]

@iritkatriel thank you reviewing this and getting it merged. @gaogaotiantian thank you for taking up this implementation.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit 44f9a84.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/7401) and take a look at the build logs.
4. Check if the failure is related to this commit (44f9a84) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/7401

Failed tests:

• test_tokenize
• test_io
• test.test_concurrent_futures.test_thread_pool

Failed subtests:

• test_normalization - test.test_unicodedata.NormalizationTest.test_normalization
• test_choices_algorithms - test.test_random.MersenneTwister_TestBasicOps.test_choices_algorithms
• test_bug_9025 - test.test_random.MersenneTwister_TestBasicOps.test_bug_9025
• test_framing_many_objects - test.test_pickletools.OptimizedPickleTests.test_framing_many_objects
• test_threads - test.test_io.PyBufferedReaderTest.test_threads
• test_sample_counts_equivalence - test.test_random.MersenneTwister_TestBasicOps.test_sample_counts_equivalence

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 882, in test_choices_algorithms
c =self.gen.choices(range(n), cum_weights=range(1, n+1), k=10000)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 492, in choicesreturn [population[bisect(cum_weights, random() * total, 0, hi)]
~~~~~~^^TypeError: 'Random' object is not callable
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_io.py", line 1574, in test_threads
random.shuffle(l)
~~~~~~~~~~~~~~^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 360, in shuffle
j = randbelow(i +1)
~~~~~~~~~^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 251, in _randbelow_with_getrandbitswhile r >= n:
^^^^^^TypeError: '>=' not supported between instances of 'Random' and 'Random'
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 409, in test_bug_9025
k =sum(randrange(6755399441055744) %3==2for i inrange(n))
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 409, in <genexpr>
k =sum(randrange(6755399441055744) %3==2for i inrange(n))
~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 313, in randrangeraiseValueError("empty range for randrange()")
ValueError: empty range for randrange()
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_io.py", line 1574, in test_threads
random.shuffle(l)
~~~~~~~~~~~~~~^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 360, in shuffle
j = randbelow(i +1)
~~~~~~~~~^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 252, in _randbelow_with_getrandbits
r = getrandbits(k)
~~~~~~~~~~~^^^TypeError: 'Random' object is not callable
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/pickletester.py", line 2617, in test_framing_many_objectsself.check_frame_opcodes(pickled)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/pickletester.py", line 2566, in check_frame_opcodesfor op, arg, pos in pickletools.genops(pickled):
...<27 lines>...
frame_end = pos +9+ arg
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/pickletools.py", line 2285, in _genopsraiseValueError("at position %s, opcode %r unknown"% (
"<unknown>"if pos isNoneelse pos,
code))
ValueError: at position 141121, opcode b'M' unknown
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 198, in _run_module_as_mainreturn _run_code(code, main_globals, None,
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^"__main__", mod_spec)
^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 88, in _run_codeexec(code, run_globals)
~~~~^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/worker.py", line 6, in <module>from test.support import os_helper, Py_DEBUG
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 15, in <module>import textwrap
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/textwrap.py", line 416, in <module>
_whitespace_only_re = re.compile('^[ \t]+$', re.MULTILINE)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/__init__.py", line 289, in compilereturn _compile(pattern, flags)
~~~~~~~~^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/__init__.py", line 350, in _compile
p = _compiler.compile(pattern, flags)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_compiler.py", line 743, in compile
p = _parser.parse(p, flags)
~~~~~~~~~~~~~^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 980, in parse
p = _parse_sub(source, state, flags &SRE_FLAG_VERBOSE, 0)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 459, in _parse_sub
itemsappend(_parse(source, state, verbose, nested +1,
~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^not nested andnot items))
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 886, in _parse
op, av = subpattern[i]
^^^^^^ValueError: not enough values to unpack (expected 2, got 0)
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 963, in test_sample_counts_equivalence
s1 = sample(colors, counts=counts, k=k)
~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 429, in sample
selections =self.sample(range(total), k=k)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 445, in sample
result[i] = pool[j]
~~~~^^^TypeError: list indices must be integers or slices, not Random
Traceback (most recent call last):
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_unicodedata.py", line 393, in test_normalizationself.run_normalization_tests(testdata)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_unicodedata.py", line 426, in run_normalization_testsself.assertTrue(c5 == NFD(c4) == NFD(c5), line)
~~~^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 690, in __call__returnself.run(*args, **kwds)
~~~~~~~~^^^^^^^^^^^^^^^
File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 609, in run
result.startTest(self)
^^^^^^^^^^^^^^^^AttributeError: 'str' object has no attribute 'startTest'

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 MacOS M1 Refleaks NoGIL 3.x has failed when building commit 44f9a84.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1368/builds/460) and take a look at the build logs.
4. Check if the failure is related to this commit (44f9a84) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1368/builds/460

Failed tests:

• test.test_asyncio.test_server

Failed subtests:

• test_abort_clients - test.test_asyncio.test_server.TestServer2.test_abort_clients

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/async_case.py", line 93, in _callTestMethodifself._callMaybeAsync(method) isnotNone:
~~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/async_case.py", line 115, in _callMaybeAsyncreturnself._asyncioRunner.run(
~~~~~~~~~~~~~~~~~~~~~~~^
func(*args, **kwargs),
^^^^^^^^^^^^^^^^^^^^^^
context=self._asyncioTestContext,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/asyncio/runners.py", line 118, in runreturnself._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/asyncio/base_events.py", line 721, in run_until_completereturn future.result()
~~~~~~~~~~~~~^^
File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_asyncio/test_server.py", line 231, in test_abort_clients
s_sock = s_wr.get_extra_info('socket')
^^^^^^^^^^^^^^^^^^^AttributeError: 'NoneType' object has no attribute 'get_extra_info'

[gaogaotiantian]

Could backporting break working code?

I don't think so. This change only affects .pdbrc and -c. The changes it made will not break any reasonable code. I can't think of any case that can be broken even if it works in a weird way.

[miss-islington-app]

Thanks @gaogaotiantian for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[miss-islington-app]

Thanks @gaogaotiantian for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @gaogaotiantian and @iritkatriel, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44f9a84b67c97c94f0d581ffd63b24b73fb79610 3.11

[miss-islington-app]

Sorry, @gaogaotiantian and @iritkatriel, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44f9a84b67c97c94f0d581ffd63b24b73fb79610 3.12

[bedevere-app]

GH-116660 is a backport of this pull request to the 3.11 branch.

[bedevere-app]

GH-116661 is a backport of this pull request to the 3.12 branch.