fix: allow Protobuf 6.x

[parthea]

See https://pypi.org/project/protobuf/6.30.0/
See googleapis/gapic-generator-python#2347 for removing dev in setup.py

Wait for googleapis/gapic-generator-python#2352, googleapis/python-api-core#804, googleapis/proto-plus-python#536

BEGIN_COMMIT_OVERRIDE
fix: add trove classifier for Python 3.13
fix: resolve issue where pre-release versions of dependencies are installed
fix(deps): require protobuf >= 3.20.2, < 7.0.0
fix(deps): require google-crc32c >= 1.1.3
fix(deps): require requests >= 2.22.0
tests: add tests for Python 3.13
tests: add pre-release testing via prerelease_deps nox session
tests(samples): add configuration for python 3.13
tests: add constraints file to test the minimum supported version of a dependency
END_COMMIT_OVERRIDE

[parthea]

Wait for cl/736096543

[parthea]

Wait for cl/736277031

[parthea]

Python 3.13 google-crc32 related tests are failing:

nox > Running session unit-3.13
nox > Creating virtual environment (virtualenv) using python3.13 in .nox/unit-3-13
nox > python -m pip install mock pytest pytest-cov brotli -c /tmpfs/src/github/python-storage/testing/constraints-3.13.txt
nox > python -m pip install opentelemetry-api opentelemetry-sdk
nox > python -m pip install -e . -c /tmpfs/src/github/python-storage/testing/constraints-3.13.txt
nox > pip uninstall -y google-resumable-media
Found existing installation: google-resumable-media 2.7.2
Uninstalling google-resumable-media-2.7.2:
Successfully uninstalled google-resumable-media-2.7.2
nox > py.test --quiet --junitxml=unit_3.13_sponge_log.xml --cov=google.cloud.storage --cov=google.cloud --cov=tests.unit --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=0 tests/unit tests/resumable_media/unit
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 34%]
......................Exception ignored in: <google.cloud.storage.fileio.BlobWriter object at 0x7f8d7cf21e10>
Traceback (most recent call last):
File "/tmpfs/src/github/python-storage/google/cloud/storage/fileio.py", line 419, in close
self._upload_chunks_from_buffer(1)
File "/tmpfs/src/github/python-storage/google/cloud/storage/fileio.py", line 387, in _upload_chunks_from_buffer
self._initiate_upload()
File "/tmpfs/src/github/python-storage/google/cloud/storage/fileio.py", line 372, in _initiate_upload
self._upload_and_transport = self._blob._initiate_resumable_upload(
File "/tmpfs/src/github/python-storage/google/cloud/storage/blob.py", line 2141, in _initiate_resumable_upload
info = self._get_upload_arguments(client, content_type, command=command)
File "/tmpfs/src/github/python-storage/google/cloud/storage/blob.py", line 1833, in _get_upload_arguments
**client._extra_headers,
File "/tmpfs/src/github/python-storage/.nox/unit-3-13/lib/python3.13/site-packages/mock/mock.py", line 699, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute '_extra_headers'
.................................................. [ 39%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 54%]
........................................................................ [ 59%]
........................................................................ [ 64%]
........................................................................ [ 69%]
........................................................................ [ 74%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 89%]
.......................F.....................................F.F........ [ 94%]
.F....................................................................F. [ 99%]
.. [100%]
=================================== FAILURES ===================================
______________________ test__is_crc32c_available_and_fast ______________________
def test__is_crc32c_available_and_fast():
import sys
import google_crc32c
> assert google_crc32c.implementation == "c"
E AssertionError: assert 'python' == 'c'
E E - c
E + python
tests/resumable_media/unit/test__helpers.py:183: AssertionError
________________ TestMultipartUpload.test_constructor_defaults _________________
self = <tests.resumable_media.unit.test__upload.TestMultipartUpload object at 0x7f8d7c737610>
def test_constructor_defaults(self):
upload = _upload.MultipartUpload(MULTIPART_URL)
assert upload.upload_url == MULTIPART_URL
assert upload._headers == {}
> assert upload._checksum_type == "crc32c" # converted from "auto"
E AssertionError: assert 'md5' == 'crc32c'
E E - crc32c
E + md5
tests/resumable_media/unit/test__upload.py:194: AssertionError
______________ TestMultipartUpload.test_constructor_explicit_auto ______________
self = <tests.resumable_media.unit.test__upload.TestMultipartUpload object at 0x7f8d7c3b49d0>
def test_constructor_explicit_auto(self):
headers = {"spin": "doctors"}
upload = _upload.MultipartUpload(
MULTIPART_URL, headers=headers, checksum="auto"
)
assert upload.upload_url == MULTIPART_URL
assert upload._headers is headers
> assert upload._checksum_type == "crc32c"
E AssertionError: assert 'md5' == 'crc32c'
E E - crc32c
E + md5
tests/resumable_media/unit/test__upload.py:214: AssertionError
_____________________ TestResumableUpload.test_constructor _____________________
self = <tests.resumable_media.unit.test__upload.TestResumableUpload object at 0x7f8d7c737750>
def test_constructor(self):
chunk_size = ONE_MB
upload = _upload.ResumableUpload(RESUMABLE_URL, chunk_size)
assert upload.upload_url == RESUMABLE_URL
assert upload._headers == {}
assert not upload._finished
_check_retry_strategy(upload)
assert upload._chunk_size == chunk_size
assert upload._stream is None
assert upload._content_type is None
assert upload._bytes_uploaded == 0
assert upload._bytes_checksummed == 0
assert upload._checksum_object is None
assert upload._total_bytes is None
assert upload._resumable_url is None
> assert upload._checksum_type == "crc32c" # converted from "auto"
E AssertionError: assert 'md5' == 'crc32c'
E E - crc32c
E + md5
tests/resumable_media/unit/test__upload.py:359: AssertionError
______________________________ test_xml_mpu_part _______________________________
filename = '/tmp/tmp6jsl5w9j'
def test_xml_mpu_part(filename):
PART_NUMBER = 1
START = 0
END = 256
ETAG = PARTS[1]
part = _upload.XMLMPUPart(
EXAMPLE_XML_UPLOAD_URL,
UPLOAD_ID,
filename,
START,
END,
PART_NUMBER,
headers=EXAMPLE_HEADERS,
checksum="md5",
)
assert part.upload_url == EXAMPLE_XML_UPLOAD_URL
assert part.upload_id == UPLOAD_ID
assert part.filename == filename
assert part.etag is None
assert part.start == START
assert part.end == END
assert part.part_number == PART_NUMBER
assert part._headers == EXAMPLE_HEADERS
assert part._checksum_type == "md5"
assert part._checksum_object is None
part = _upload.XMLMPUPart(
EXAMPLE_XML_UPLOAD_URL,
UPLOAD_ID,
filename,
START,
END,
PART_NUMBER,
headers=EXAMPLE_HEADERS,
checksum="auto",
)
assert part.upload_url == EXAMPLE_XML_UPLOAD_URL
assert part.upload_id == UPLOAD_ID
assert part.filename == filename
assert part.etag is None
assert part.start == START
assert part.end == END
assert part.part_number == PART_NUMBER
assert part._headers == EXAMPLE_HEADERS
> assert part._checksum_type == "crc32c" # transformed from "auto"
E AssertionError: assert 'md5' == 'crc32c'
E E - crc32c
E + md5
tests/resumable_media/unit/test__upload.py:1425: AssertionError
=============================== warnings summary ===============================
.nox/unit-3-13/lib/python3.13/site-packages/google_crc32c/__init__.py:29
/tmpfs/src/github/python-storage/.nox/unit-3-13/lib/python3.13/site-packages/google_crc32c/__init__.py:29: RuntimeWarning: As the c extension couldn't be imported, `google-crc32c` is using a pure python implementation that is significantly slower. If possible, please configure a c build environment and compile the extension
warnings.warn(_SLOW_CRC32C_WARNING, RuntimeWarning)

[parthea]

The issue is that wheels aren't published for google-crc32 yet

[parthea]

@cojenco, Please could you take a look?

[cojenco]

Thanks for working on this!

[cojenco]

IIUC this is setting a list of sessions to run by default (to exclude prerelease_deps so that it is only ran in kokoro prerelease_deps)

[parthea]

Yes, this is correct. Also see https://nox.thea.codes/en/stable/tutorial.html#selecting-which-sessions-to-run

[cojenco]

I'm wondering if we've considered running prerelease-deps in continuous instead of presubmits. This increases presubmit running time and the extra two runs of the system test suite are more prone to 429 errors and server-side resource overload etc.

[parthea]

Fixed in cl/738167833 and 77a3dc8

[github-advanced-security]

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.