IDLE: Colorizer and output tests hang on macOS

[terryjreedy]

BPO	44330
Nosy	@terryjreedy, @taleinat, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-06-07.01:41:11.947>
labels = ['3.9', '3.10', '3.11']
title = 'IDLE: Colorizer and output tests hang on macOS'
updated_at = <Date 2021-06-07.22:11:14.604>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2021-06-07.22:11:14.604>
actor = 'terry.reedy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-06-07.01:41:11.947>
creator = 'terry.reedy'
dependencies = []
files = []
hgrepos = []
issue_num = 44330
keywords = []
message_count = 3.0
messages = ['395239', '395285', '395290']
nosy_count = 3.0
nosy_names = ['terry.reedy', 'taleinat', 'ned.deily']
pr_nums = []
priority = 'critical'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue44330'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• gh-88496: Fix IDLE test hang on macOS #104025
• [3.11] gh-88496: Fix IDLE test hang on macOS (GH-104025) #104027
• gh-88496: IDLE - fix another test on macOS #104075
• [3.11] gh-88496: IDLE - fix another test on macOS (GH-104075) #104076

[terryjreedy]

Ned, have you seen or heard anything about tkinter/tk8.6.11 freezing on macOS? It is happening in many places.

On my MacBook, python3.9 -m test -ugui -v test_idle runs OK in about 10 seconds. With installed python3.10.0b2, it runs test_colorizer ColorCongigTest, ColorDelagatorInstantionTest, and ColorDelagatorTest.test_LoadTagDefs. It hangs on test_case_soft_keyword, which is new in 3.10 for the new match statement soft keywords. With added skips, it hangs on
test_def_statement
test_incremental_editing
test_match_soft_keyword
test_recolorize_main
test_removecolors
and then test_colorizer passes.

At this point, I thought the problem was the fancy new test methods and helpers. But running test_idle again, test_outwin.OutputWindowTest.test_goto_file_line (line 105) hangs displaying a blank window. The test writes something at 115 which did not appear. However, debug prints indicate that it is the 2nd write, about 121, that does not return. The write method just inserts into the text widget. Nothing apparently fancy. The failure is deterministic (over 10 times). In 3.9.5, the same test and all of test_idle ran OK 3 times.

Adding .update() and .update_idletasks() before the write did nothing. Making this test method fail before the write, let the test continue until it hung in test_write.

The problem seems to be in the test environment. Tests of match, case, and _ in Shell looked corrected. Grepping idlelib for 'tkinter' returned 222 hits and multiple gotos for different files and lines within a file worked normally.

[ned-deily]

I can reproduce test_idle hanging on all of the current python.org macOS universal2 variants (3.8.10, 3.9.5, 3.10.0b2) which use Tk 8.6.11 but not with the legacy 10.9 variants for 3.8.10 and 3.9.5 which use Tk 8.6.8. I've tried it on a few older systems, as far back as 10.9, and they all seem to hang the same way. For 3.9.5 (and 3.8.10 now in security-fix mode), the first hang seems to be in test_removecolors which I need to Ctrl-C out of:

/usr/local/bin/python3.9 -m test -v -uall test_idle
[...]
test_insert (idlelib.idle_test.test_colorizer.ColorDelegatorTest) ... ok
test_notify_range (idlelib.idle_test.test_colorizer.ColorDelegatorTest) ... ok
test_recolorize (idlelib.idle_test.test_colorizer.ColorDelegatorTest) ... ok
test_recolorize_main (idlelib.idle_test.test_colorizer.ColorDelegatorTest) ... ok
test_removecolors (idlelib.idle_test.test_colorizer.ColorDelegatorTest) ...

I also see the hangs with current MacPorts Pythons linked with their build of Tk 8.6.11.

[terryjreedy]

Yes, I have the 'old' 3.9.5. In IDLE, ^C does not break the hang, but Restart Shell does. As I expected, I had to skip test_outwin's test_write and test_writelines to stop that test hanging.

Next failure: test_sidebar.LineNumbersTest ends with test_toggle_linenumbering ... OK. That is the last test method and the hang is in tearDownClass "cls.root.update()". Changing to .update_idletasks() fixes this. I believe that the latter is what we usually use before destroying root. But still, update should not crash. At worst, a TclError.

With this fix, test_idle passes. So do test_tcl and test_ttk_textonly. Test_tk and test_ttk_guionly each have 1 failure. (For me) The test suite as a whole is a mess, with multiple Python-crashed message boxes.