gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. by CorvinM · Pull Request #108657 · python/cpython

CorvinM · 2023-08-29T21:01:10Z

Fixes an exception where sqlite3.iterdump would fail to decode if someone stuffed invalid unicode into a database column marked as TEXT (or alias of, like VARCHAR).
See #108590 for more info

Issue: sqlite3.iterdump() incompatible with binary data #108590

…umns.

…k0pOl.rst Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

erlend-aasland

Thanks for doing this, Corvin; it is a nice solution to this corner case.

I'll make some final adjustments before landing.

erlend-aasland · 2023-08-30T08:37:32Z

@CorvinM, you may also be interested in looking at #108364

CorvinM · 2023-08-30T08:39:52Z

Thank you for the speedy reviews and helpful feedback 😄 Let me know if you want me to pick up anything else with this (e.g., backports)

miss-islington · 2023-08-30T09:06:27Z

Sorry, @CorvinM and @erlend-aasland, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 400a1cebc743515e40157ed7af86e48d654290ce 3.12

miss-islington · 2023-08-30T09:06:30Z

Sorry, @CorvinM and @erlend-aasland, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 400a1cebc743515e40157ed7af86e48d654290ce 3.11

erlend-aasland · 2023-08-30T09:07:50Z

@CorvinM, can you handle the backports? You can use the cherry-picker. See the devguide.

CorvinM · 2023-08-30T09:21:01Z

@CorvinM, can you handle the backports? You can use the cherry-picker. See the devguide.

Sure

…EXT columns (pythonGH-108657) Co-authored-by: Erlend E. Aasland <erlend@python.org>. (cherry picked from commit 400a1ce) Co-authored-by: Corvin <corvin@corvin.dev>

bedevere-bot · 2023-08-30T09:46:38Z

GH-108673 is a backport of this pull request to the 3.12 branch.

…EXT columns (pythonGH-108657) Co-authored-by: Erlend E. Aasland <erlend@python.org>. (cherry picked from commit 400a1ce) Co-authored-by: Corvin <corvin@corvin.dev>

bedevere-bot · 2023-08-30T09:56:27Z

GH-108674 is a backport of this pull request to the 3.11 branch.

…lumns (GH-108657) (#108674) (cherry picked from commit 400a1ce)

Reverted per Serhiy's request.

* main: pythongh-108520: Fix bad fork detection in nested multiprocessing use case (python#108568) pythongh-108590: Revert pythongh-108657 (commit 400a1ce) (python#108686) pythongh-108494: Argument Clinic: Document how to generate code that uses the limited C API (python#108584) Document Python build requirements (python#108646) pythongh-101100: Fix Sphinx warnings in the Logging Cookbook (python#108678) Fix typo in multiprocessing docs (python#108666) pythongh-108669: unittest: Fix documentation for TestResult.collectedDurations (python#108670) pythongh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (python#108657) Revert "pythongh-103224: Use the realpath of the Python executable in `test_venv` (pythonGH-103243)" (pythonGH-108667) pythongh-106320: Remove private _Py_ForgetReference() (python#108664) Mention Ellipsis pickling in the docs (python#103660) Revert "Use non alternate name for Kyiv (pythonGH-108533)" (pythonGH-108649) pythongh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (python#108632) pythongh-108455: peg_generator: install two stubs packages before running mypy (python#108637) pythongh-107801: Improve the accuracy of io.IOBase.seek docs (python#108268)

…on#108686) (cherry picked from commit 2a3926f) Reverted per Serhiy's request.

(cherry picked from commit 2a3926f) Reverted per Serhiy's request.

pythongh-108590: Fix sqlite3.iterdump for invalid unicode in text col…
a689bbd
…umns.

bedevere-bot mentioned this pull request Aug 29, 2023
sqlite3.iterdump() incompatible with binary data #108590
Closed
2 tasks

CorvinM commented Aug 29, 2023
View reviewed changes

Comment threadLib/sqlite3/dump.py

CorvinM marked this pull request as ready for review August 29, 2023 22:49

CorvinM requested review from berkerpeksag and erlend-aasland as code owners August 29, 2023 22:49

bedevere-bot added the awaiting review label Aug 29, 2023

📜🤖 Added by blurb_it.
03f41df

erlend-aasland reviewed Aug 30, 2023
View reviewed changes

Comment threadLib/sqlite3/dump.py Outdated
Comment threadMisc/NEWS.d/next/Library/2023-08-29-22-53-48.gh-issue-108590.6k0pOl.rst Outdated
Comment threadLib/sqlite3/dump.py Outdated

CorvinMand others added 3 commits August 30, 2023 03:41

Update Misc/NEWS.d/next/Library/2023-08-29-22-53-48.gh-issue-108590.6…
e2939d5
…k0pOl.rst Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Update Lib/sqlite3/dump.py
0643ffc
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Address comments
608d919

erlend-aasland reviewed Aug 30, 2023
View reviewed changes

Comment threadLib/sqlite3/dump.py Outdated

Use contextmanager
cc7fba8

erlend-aasland approved these changes Aug 30, 2023
View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Aug 30, 2023

Style nits and a comment
c1b730d

erlend-aasland enabled auto-merge (squash) August 30, 2023 08:28

erlend-aasland added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Aug 30, 2023

erlend-aasland linked an issue Aug 30, 2023 that may be closed by this pull request
sqlite3.iterdump() incompatible with binary data #108590
Closed
2 tasks

erlend-aasland merged commit 400a1ce into python:mainAug 30, 2023

This comment was marked as outdated.
Sign in to view

bedevere-bot removed the awaiting merge label Aug 30, 2023

miss-islington assigned erlend-aasland Aug 30, 2023

CorvinM mentioned this pull request Aug 30, 2023
[3.12] gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (GH-108657) #108673
Closed

bedevere-bot removed the needs backport to 3.12 only security fixes label Aug 30, 2023

CorvinM mentioned this pull request Aug 30, 2023
[3.11] gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (GH-108657) #108674
Merged

bedevere-bot removed the needs backport to 3.11 only security fixes label Aug 30, 2023

erlend-aasland pushed a commit that referenced this pull request Aug 30, 2023
[3.11]gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT co…
5a6d123
…lumns (GH-108657) (#108674) (cherry picked from commit 400a1ce)

erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Aug 30, 2023
pythongh-108590: Revertpythongh-108657(commit400a1ce)
cd05820
Reverted per Serhiy's request.

erlend-aasland added a commit that referenced this pull request Aug 30, 2023
gh-108590: Revertgh-108657(commit400a1ce) (#108686)
2a3926f
Reverted per Serhiy's request.

erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Aug 30, 2023
[3.11]pythongh-108590: Revertpythongh-108657(commit400a1ce) (pyth…
8888729
…on#108686) (cherry picked from commit 2a3926f) Reverted per Serhiy's request.

erlend-aasland added a commit that referenced this pull request Aug 30, 2023
[3.11]gh-108590: Revertgh-108657(commit400a1ce) (#108686) (#108694)
6f24420
(cherry picked from commit 2a3926f) Reverted per Serhiy's request.

Navigation Menu

Uh oh!

Uh oh!

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. - #108657

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns.#108657
erlend-aasland merged 7 commits into
python:mainfrom
CorvinM:gh-108590

CorvinM commented Aug 29, 2023•
edited
Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erlend-aasland left a comment

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

This comment was marked as outdated.

miss-islington commented Aug 30, 2023

Uh oh!

miss-islington commented Aug 30, 2023

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Uh oh!

Conversation

CorvinM commented Aug 29, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erlend-aasland left a comment

Choose a reason for hiding this comment

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

This comment was marked as outdated.

miss-islington commented Aug 30, 2023

Uh oh!

miss-islington commented Aug 30, 2023

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CorvinM commented Aug 29, 2023•
edited
Loading