Skip to content

json: improve bytes handling - #9042

Merged
JelleZijlstra merged 1 commit into
python:masterfrom
sobolevn:bytes-json
Oct 30, 2022
Merged

json: improve bytes handling#9042
JelleZijlstra merged 1 commit into
python:masterfrom
sobolevn:bytes-json

Conversation

@sobolevn

@sobolevnsobolevn commented Oct 30, 2022

Copy link
Copy Markdown
Member

dumps and detect_encoding support bytearray:

>>>importjson>>>json.loads(bytearray(b'false'))
False>>>json.detect_encoding(bytearray(b''))
'utf-8'

But, not memoryview:

>>>json.loads(memoryview(b'false'))
Traceback (mostrecentcalllast):
File"<stdin>", line1, in<module>File"/Users/sobolev/.pyenv/versions/3.10.0/lib/python3.10/json/__init__.py", line339, inloadsraiseTypeError(f'the JSON object must be str, bytes or bytearray, 'TypeError: theJSONobjectmustbestr, bytesorbytearray, notmemoryview

@github-actions

Copy link
Copy Markdown
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

optuna (https://github.com/optuna/optuna)
- optuna/storages/_rdb/storage.py:388: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes]"+ optuna/storages/_rdb/storage.py:388: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes, bytearray]"- optuna/storages/_rdb/storage.py:398: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes]"+ optuna/storages/_rdb/storage.py:398: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes, bytearray]"- optuna/storages/_rdb/storage.py:409: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes]"+ optuna/storages/_rdb/storage.py:409: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes, bytearray]"- optuna/storages/_rdb/storage.py:420: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes]"+ optuna/storages/_rdb/storage.py:420: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes, bytearray]"- optuna/storages/_rdb/storage.py:454: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes]"+ optuna/storages/_rdb/storage.py:454: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes, bytearray]"- optuna/storages/_rdb/storage.py:455: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes]"+ optuna/storages/_rdb/storage.py:455: error: Argument 1 to "loads" has incompatible type "Column"; expected "Union[str, bytes, bytearray]"

@hauntsaninjahauntsaninja left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@JelleZijlstraJelleZijlstra left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loads explicitly checks for str | bytes | bytearray. detect_encoding does .startswith which doesn't exist on any other well-known buffer type.

@JelleZijlstra
JelleZijlstra merged commit 6569dea into python:masterOct 30, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@sobolevn@JelleZijlstra@hauntsaninja