Version 3.3.1 has broken blob.upload_from_file(file_obj) with error: Provided CRC32C xMvOLw== doesn't match calculated CRC32C ObqI+Q==

[dougEdwards]

python version 3.10
gcp-cloud run application

Version 3.3.1 of google-cloud-storage

blob.upload_from_file(file_obj)

error thrown:

"Provided CRC32C xMvOLw== doesn't match calculated CRC32C ObqI+Q=="

workaround: revert to Version 3.3.0 or earlier
e.g
"google-cloud-storage>=3.0.0,<3.3.1",

also blob.upload_from_filename() has the same problem

[chandra-siri]

Hi @dougEdwards , in version 3.3.1. support was added to include CRC32C checksum of an object. i.e. users can now provide crc32c checksum (calculated by them) while initializing the blob.

This checksum will be matched by the checksum calculated by server, if they don't match, server will send an error. Which is what is happening.

Sample to send checksum of the object while initializing

 storage_client = storage.Client()
 bucket = storage_client.bucket(bucket_name)
 blob = bucket.blob(
 destination_blob_name, crc32c_checksum="your_calculated_checksum"
 )

 # Optional: set a generation-match precondition to avoid potential race conditions
 # and data corruptions. The request to upload is aborted if the object's
 # generation number does not match your precondition. For a destination
 # object that does not yet exist, set the if_generation_match precondition to 0.
 # If the destination object already exists in your bucket, set instead a
 # generation-match precondition using its generation number.
 generation_match_precondition = 0

 blob.upload_from_filename(
 source_file_name, if_generation_match=generation_match_precondition
 )

 print(f"File {source_file_name} uploaded to {destination_blob_name}.")

Could you also provide us with a sample to replicated the behaviour you're observing ?

[chrisaiello]

@chandra-siri

I'm encountering the same/similar issue via django-storages

The pattern that reproduces the error appears to be:

• Create an empty file (probably doesn't have to be empty)
• blob.reload() is called, which populates the crc32c checksum property
• Attempt to overwrite the file

Here's a reproducible example that succeeds in v3.3.0 but fails in v3.3.1

https://gist.github.com/chrisaiello/6c393cbbb96bb93db9e22ac24c1c391a

Details

Using blob: gs://my_bucket/bug-repro-0d15ff63a6384cecab37a52bf30a06a6.dat
Step 1: Uploading an empty file...
Step 1: Success.

Step 2: Reloading blob metadata from server...
Step 2: Success. Local blob object now has crc32c = 'AAAAAA=='

Step 3: Attempting to overwrite the blob with new data (9437184 bytes)...

--- ERROR REPRODUCED ---
Caught expected google.api_core.exceptions.BadRequest
Status Code: 400
Error Message: PUT https://storage.googleapis.com/upload/storage/v1/b/my_bucket/o?uploadType=resumable&upload_id=ABgVH8--9DCW8DHjWO8XTI7WzQ4fnJfXenfkdZqdh6RlCoocrpjb9VtjSuxESRYEdSML-OKa3xKf6iKCSH4ifDtYjBcI8OgweuTSfyDrdDnG7c5I: {
 "error": {
 "code": 400,
 "message": "Provided CRC32C \"AAAAAA==\" doesn't match calculated CRC32C \"Gs/MBw==\".",
 "errors": [
 {
 "message": "Provided CRC32C \"AAAAAA==\" doesn't match calculated CRC32C \"Gs/MBw==\".",
 "domain": "global",
 "reason": "invalid"
 }
 ]
 }
}
: ('Request failed with status code', 400, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PERMANENT_REDIRECT: 308>)
-----------------------------------

[jacobg]

Yep me too! Since updating from version 3.3.0 to 3.4.0, I'm also seeing lots of these errors in production:

400 PUT https://storage.googleapis.com/upload/storage/v1/b/f-reports/o?uploadType=resumable&upload_id={{REDACTED}: {
 "error": {
 "code": 400,
 "message": "Provided CRC32C \"AAAAAA==\" doesn't match calculated CRC32C \"2HvlZg==\".",
 "errors": [
 {
 "message": "Provided CRC32C \"AAAAAA==\" doesn't match calculated CRC32C \"2HvlZg==\".",
 "domain": "global",
 "reason": "invalid"
 }
 ]
 }

 File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 3130, in upload_from_string
 self.upload_from_file(
 ^
 File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 2864, in upload_from_file
 self._prep_and_do_upload(
 ^
 File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 2735, in _prep_and_do_upload
 _raise_from_invalid_response(exc)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 4925, in _raise_from_invalid_response
 raise exceptions.from_http_status(response.status_code, message, response=response)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[jacobg]

@chandra-siri Do you have an update on a fix for the regression caused in version 3.3.1?

[chandra-siri]

Hi all,

We'll soon be fixing this issue, you should see the fix rolled out in < 2 weeks.

Thanks @chrisaiello for providing with the repro script, & thanks @dougEdwards for reporting the bug.

[chandra-siri]

Hi @jacobg , @chrisaiello and @dougEdwards

This issue is now fixed. Please upgrade to the latest version v3.4.1