gh-115133: Fix tests for XMLPullParser with Expat 2.6.0 by serhiy-storchaka · Pull Request #115164 · python/cpython

serhiy-storchaka · 2024-02-08T12:25:55Z

Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

Issue: test.test_xml_etree*.XMLPullParserTest.test_simple_xml fails with (system) expat 2.6.0 #115133

Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

lazka · 2024-02-10T17:43:38Z

It stills fails here with this patch applied:

FAIL: test_simple_xml_chunk_8 (test.test_xml_etree.XMLPullParserTest.test_simple_xml_chunk_8)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\a\cpython-mingw\cpython-mingw\Lib\test\test_xml_etree.py", line 1438, in test_simple_xml_chunk_8
self.test_simple_xml(chunk_size=8)
File "D:\a\cpython-mingw\cpython-mingw\Lib\test\test_xml_etree.py", line 1418, in test_simple_xml
self.assert_event_tags(parser, [('end', 'element')])
File "D:\a\cpython-mingw\cpython-mingw\Lib\test\test_xml_etree.py", line 1405, in assert_event_tags
self.assertEqual([(action, elem.tag) for action, elem in events],
AssertionError: Lists differ: [] != [('end', 'element')]
Second list contains 1 additional elements.
First extra element 0:
('end', 'element')
- []
+ [('end', 'element')]

> python3 -c "import pyexpat; print(pyexpat.version_info)"
(2, 6, 0)

serhiy-storchaka · 2024-02-10T18:06:05Z

What is the smallest value of chunk_size with which the test would pass?

lazka · 2024-02-10T18:10:43Z

chunk_size=22 is the smallest value that works on my machine.

serhiy-storchaka · 2024-02-10T18:50:55Z

Thank you for testing @lazka.

miss-islington-app · 2024-02-11T10:08:42Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-app · 2024-02-11T10:08:51Z

GH-115288 is a backport of this pull request to the 3.12 branch.

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-app · 2024-02-11T10:08:57Z

GH-115289 is a backport of this pull request to the 3.11 branch.

…5164) (GH-115288) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…5164) (GH-115289) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

miss-islington-app · 2024-02-15T15:21:25Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

miss-islington-app · 2024-02-15T15:21:25Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

miss-islington-app · 2024-02-15T15:21:25Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington-app · 2024-02-15T15:21:28Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a08e7b3431cd32a0daf22a33421cd3035343dc4 3.8

miss-islington-app · 2024-02-15T15:21:44Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a08e7b3431cd32a0daf22a33421cd3035343dc4 3.9

…GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-app · 2024-02-15T15:22:08Z

GH-115525 is a backport of this pull request to the 3.10 branch.

…ythonGH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

sethmlarson · 2024-02-15T20:23:41Z

Created backports for 3.9 and 3.8 manually:

…5164) (#115525) gh-115133: Fix tests for XMLPullParser with Expat 2.6.0 (GH-115164) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…ythonGH-115164) (pythonGH-115288) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

) (GH-115536) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…ythonGH-115164) (pythonGH-115288) Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive. (cherry picked from commit 4a08e7b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

pythongh-115133: Fix tests for XMLPullParser with Expat 2.6.0
e27eafb
Feeding the parser by too small chunks defers parsing to prevent CVE-2023-52425. Future versions of Expat may be more reactive.

serhiy-storchaka added tests Tests in the Lib/test dir needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Feb 8, 2024

bedevere-appBot added the awaiting core review label Feb 8, 2024

bedevere-appBot mentioned this pull request Feb 8, 2024
test.test_xml_etree*.XMLPullParserTest.test_simple_xml fails with (system) expat 2.6.0 #115133
Closed

Snild-Sony approved these changes Feb 8, 2024
View reviewed changes

hartwork approved these changes Feb 10, 2024
View reviewed changes

serhiy-storchaka commented Feb 10, 2024
View reviewed changes

Comment threadLib/test/test_xml_etree.py Outdated

Update Lib/test/test_xml_etree.py
aaffd5d

serhiy-storchaka merged commit 4a08e7b into python:mainFeb 11, 2024

bedevere-appBot removed the awaiting core review label Feb 11, 2024

serhiy-storchaka deleted the test-etree-xmlpullparser-expat-2.6.0 branch February 11, 2024 10:08

bedevere-appBot removed the needs backport to 3.12 only security fixes label Feb 11, 2024

bedevere-appBot removed the needs backport to 3.11 only security fixes label Feb 11, 2024

serhiy-storchaka mentioned this pull request Feb 11, 2024
gh-115133: test_xml_etree.py: Fix for Expat >=2.6.0 with reparse deferral (fixes #115133) #115138
Closed

hartwork mentioned this pull request Feb 12, 2024
expat: 2.5.0 -> 2.6.0 NixOS/nixpkgs#287040
Merged
13 tasks

hartwork mentioned this pull request Feb 14, 2024
Release Expat 2.6.0 libexpat/libexpat#775
Closed
28 tasks

hartwork mentioned this pull request Feb 14, 2024
[3.8] Upgrade bundled libexpat to 2.6.0 (GH-115399) #115475
Merged

sethmlarson added needs backport to 3.8 needs backport to 3.10 only security fixes labels Feb 15, 2024

miss-islington-appBot assigned serhiy-storchaka Feb 15, 2024

bedevere-appBot removed the needs backport to 3.10 only security fixes label Feb 15, 2024

Bill-Sommerfeld mentioned this pull request Feb 20, 2024
epoll patch Bill-Sommerfeld/oi-userland#13
Closed

edmorley mentioned this pull request Oct 10, 2024
Python 3.13: test_xml_etree fails on expat 2.2.5 (RHEL8 / Rocky linux 8 / Ubuntu 22.04) #125067
Closed

hugovk removed the needs backport to 3.9 label Feb 18, 2025

serhiy-storchaka removed their assignment Jul 15, 2026

Uh oh!

Uh oh!

Conversation

serhiy-storchaka commented Feb 8, 2024• edited by bedevere-app BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lazka commented Feb 10, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka commented Feb 10, 2024

Uh oh!

lazka commented Feb 10, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

serhiy-storchaka commented Feb 10, 2024

Uh oh!

miss-islington-appBot commented Feb 11, 2024

Uh oh!

bedevere-appBot commented Feb 11, 2024

Uh oh!

bedevere-appBot commented Feb 11, 2024

Uh oh!

miss-islington-appBot commented Feb 15, 2024

Uh oh!

miss-islington-appBot commented Feb 15, 2024

Uh oh!

miss-islington-appBot commented Feb 15, 2024

Uh oh!

miss-islington-appBot commented Feb 15, 2024

Uh oh!

miss-islington-appBot commented Feb 15, 2024

Uh oh!

bedevere-appBot commented Feb 15, 2024

Uh oh!

sethmlarson commented Feb 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

serhiy-storchaka commented Feb 8, 2024•
edited by bedevere-app Bot
Loading

lazka commented Feb 10, 2024•
edited
Loading

lazka commented Feb 10, 2024•
edited
Loading