gh-123678: Upgrade libexpat 2.6.3

[sethmlarson]

• Issue: Please upgrade bundled Expat to 2.6.3 (e.g. for the fixes to CVE-2024-45490, CVE-2024-45491 and CVE-2024-45492) #123678

[hartwork]

@sethmlarson verified using the Dockerfile below 👍

# Copyright (c) 2022-2024 Sebastian Pipping <sebastian@pipping.org># Licensed under the Apache License version 2.0FROM alpine
RUN apk add --update \
diffutils \
git \
sed \
&& \
git clone --depth 1 https://github.com/python/cpython cpython-main \
&& \
( cd cpython-main && git rev-parse HEAD ) \
&& \
git clone --depth 1 --branch expat-2.6.3 https://github.com/sethmlarson/cpython cpython-pr \
&& \
( cd cpython-pr && git rev-parse HEAD ) \
&& \
git config --global advice.detachedHead false \
&& \
git clone --depth 1 --branch R_2_6_2 https://github.com/libexpat/libexpat libexpat_2_6_2 \
&& \
git clone --depth 1 --branch R_2_6_3 https://github.com/libexpat/libexpat libexpat_2_6_3 \
&& \
diff -r -u libexpat_2_6_2/expat/lib/ cpython-main/Modules/expat/ | tee 2-6-2.diff \
&& \
diff -r -u libexpat_2_6_3/expat/lib/ cpython-pr/Modules/expat/ | tee 2-6-3.diff \
&& \
sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-6-2.diff 2-6-3.diff \
&& \
diff -u 2-6-2.diff 2-6-3.diff \
&& \
echo 'Diff is good.'

# docker build .DEPRECATED: The legacy builder is deprecated and will be removed in a future release. Install the buildx component to build images with BuildKit: https://docs.docker.com/go/buildx/Sending build context to Docker daemon 3.072kBStep 1/2 : FROM alpine ---> 05455a08881e
Step 2/2 : RUN apk add --update diffutils git sed && git clone --depth 1 https://github.com/python/cpython cpython-main && ( cd cpython-main && git rev-parse HEAD ) && git clone --depth 1 --branch expat-2.6.3 https://github.com/sethmlarson/cpython cpython-pr && ( cd cpython-pr && git rev-parse HEAD ) && git config --global advice.detachedHead false && git clone --depth 1 --branch R_2_6_2 https://github.com/libexpat/libexpat libexpat_2_6_2 && git clone --depth 1 --branch R_2_6_3 https://github.com/libexpat/libexpat libexpat_2_6_3 && diff -r -u libexpat_2_6_2/expat/lib/ cpython-main/Modules/expat/ | tee 2-6-2.diff && diff -r -u libexpat_2_6_3/expat/lib/ cpython-pr/Modules/expat/ | tee 2-6-3.diff && sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-6-2.diff 2-6-3.diff && diff -u 2-6-2.diff 2-6-3.diff && echo 'Diff is good.'
 ---> Running in 03fb2eed1a29fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gzfetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz(1/12) Installing diffutils (3.10-r0)(2/12) Installing ca-certificates (20240226-r0)(3/12) Installing brotli-libs (1.1.0-r1)(4/12) Installing c-ares (1.27.0-r0)(5/12) Installing libunistring (1.1-r2)(6/12) Installing libidn2 (2.3.4-r4)(7/12) Installing nghttp2-libs (1.58.0-r0)(8/12) Installing libcurl (8.9.0-r0)(9/12) Installing libexpat (2.6.2-r0)(10/12) Installing pcre2 (10.42-r2)(11/12) Installing git (2.43.4-r0)(12/12) Installing sed (4.9-r2)Executing busybox-1.36.1-r15.triggerExecuting ca-certificates-20240226-r0.triggerOK: 19 MiB in 27 packagesCloning into 'cpython-main'...Updating files: 100% (5350/5350), done.d2eafe2f48aac31aa8a152620bdfd0f2a274ee1dCloning into 'cpython-pr'...Updating files: 100% (5350/5350), done.e6fe0ac014e8f5b17064c20d1ae937dbd4c761a2Cloning into 'libexpat_2_6_2'...Cloning into 'libexpat_2_6_3'...Only in libexpat_2_6_2/expat/lib/: .gitignoreOnly in cpython-main/Modules/expat/: COPYINGOnly in libexpat_2_6_2/expat/lib/: Makefile.amOnly in cpython-main/Modules/expat/: expat_config.hdiff -r -u libexpat_2_6_2/expat/lib/expat_external.h cpython-main/Modules/expat/expat_external.h--- libexpat_2_6_2/expat/lib/expat_external.h 2024-09-04 18:04:32.429990554 +0000+++ cpython-main/Modules/expat/expat_external.h 2024-09-04 18:04:21.161886969 +0000@@ -64,6 +64,11 @@ compiled with the cdecl calling convention as the default since system headers may assume the cdecl convention. */++/* Namespace external symbols to allow multiple libexpat version to+ co-exist. */+#include "pyexpatns.h"+ #ifndef XMLCALL # if defined(_MSC_VER) # define XMLCALL __cdeclOnly in libexpat_2_6_2/expat/lib/: libexpat.def.cmakeOnly in cpython-main/Modules/expat/: pyexpatns.hOnly in libexpat_2_6_3/expat/lib/: .gitignoreOnly in cpython-pr/Modules/expat/: COPYINGOnly in libexpat_2_6_3/expat/lib/: Makefile.amOnly in cpython-pr/Modules/expat/: expat_config.hdiff -r -u libexpat_2_6_3/expat/lib/expat_external.h cpython-pr/Modules/expat/expat_external.h--- libexpat_2_6_3/expat/lib/expat_external.h 2024-09-04 18:04:35.764021202 +0000+++ cpython-pr/Modules/expat/expat_external.h 2024-09-04 18:04:29.352962267 +0000@@ -64,6 +64,11 @@ compiled with the cdecl calling convention as the default since system headers may assume the cdecl convention. */++/* Namespace external symbols to allow multiple libexpat version to+ co-exist. */+#include "pyexpatns.h"+ #ifndef XMLCALL # if defined(_MSC_VER) # define XMLCALL __cdeclOnly in libexpat_2_6_3/expat/lib/: libexpat.def.cmakeOnly in cpython-pr/Modules/expat/: pyexpatns.hDiff is good. ---> Removed intermediate container 03fb2eed1a29 ---> e85a4065bbe4Successfully built e85a4065bbe4

[miss-islington-app]

Thanks @sethmlarson for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-123707 is a backport of this pull request to the 3.13 branch.

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.11

[bedevere-app]

GH-123708 is a backport of this pull request to the 3.12 branch.

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.10

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.9

[sethmlarson]

Ah I'll close the backports that succeeded and handle the rest, thanks @gpshead!

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.8

[gpshead]

I'm not sure if this needs further backporting or not, i didn't look to see what versions those used. the specific int vs size_t change for the bugfix could likely be manually applied if relevant though. leaving that up to you.

[sethmlarson]

@gpshead Looks like every branch is upgradeable, so I went ahead and backported to every branch. Those PRs can be closed if we decide not to move forward with them.

[hartwork]

@gpshead Looks like every branch is upgradeable, so I went ahead and backported to every branch.

@sethmlarson 👍 👍