Storage: 'test_bpo_set_unset_preserves_acls' no longer sees expected 'BadRequest'.

[tseaver]

From this Kokoro job:

____________ TestIAMConfiguration.test_bpo_set_unset_preserves_acls ____________
self = <tests.system.TestIAMConfiguration testMethod=test_bpo_set_unset_preserves_acls>
 def test_bpo_set_unset_preserves_acls(self):
 new_bucket_name = "bpo-acls" + unique_resource_id("-")
 self.assertRaises(
 exceptions.NotFound, Config.CLIENT.get_bucket, new_bucket_name
 )
 bucket = retry_429(Config.CLIENT.create_bucket)(new_bucket_name)
 self.case_buckets_to_delete.append(new_bucket_name)
 blob_name = "my-blob.txt"
 blob = bucket.blob(blob_name)
 payload = b"DEADBEEF"
 blob.upload_from_string(payload)
 # Preserve ACLs before setting BPO
 bucket_acl_before = list(bucket.acl)
 blob_acl_before = list(bucket.acl)
 # Set BPO
 bucket.iam_configuration.bucket_policy_only_enabled = True
 bucket.patch()
 # While BPO is set, cannot get / set ACLs
 with self.assertRaises(exceptions.BadRequest):
> bucket.acl.reload()
E AssertionError: BadRequest not raised
tests/system.py:1709: AssertionError

[tseaver]

I'm not sure this is actually "flaky": the back-end appears to no longer return a BadRequest for bucketAccessControls.list when the bucket-policy-only flag is set.

@frankyn Can you confirm that the back-end behavior is changed?

[tseaver]

Very strange: the call to Bucket.acl.reload does raise the expected BadRequest when running under Python 2.7, but does not raise as expected under Python 3.6.

[tseaver]

Running the system test with Python 3.6, here is what I see from a breakpoint inside BucketACL.reload:

(Pdb) l
432 	 if self.user_project is not None:
433 	 query_params["userProject"] = self.user_project
434 	
435 	 self.entities.clear()
436 	
437 ->	 found = client._connection.api_request(
438 	 method="GET", path=path, query_params=query_params
439 	 )
440 	 self.loaded = True
441 	 for entry in found.get("items", ()):
442 	 self.add_entity(self.entity_from_dict(entry))
(Pdb) pp path
'/b/bpo-acls-1562617375855/acl'
(Pdb) pp query_params
{}
(Pdb) n
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(438)reload()
-> method="GET", path=path, query_params=query_params
(Pdb) 
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(440)reload()
-> self.loaded = True
(Pdb) pp found
{'items': [{'bucket': 'bpo-acls-1562617375855',
 'entity': 'project-owners-455332057385',
 'etag': 'CAI=',
 'id': 'bpo-acls-1562617375855/project-owners-455332057385',
 'kind': 'storage#bucketAccessControl',
 'projectTeam': {'projectNumber': '455332057385', 'team': 'owners'},
 'role': 'OWNER',
 'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855/acl/project-owners-455332057385'},
 {'bucket': 'bpo-acls-1562617375855',
 'entity': 'project-editors-455332057385',
 'etag': 'CAI=',
 'id': 'bpo-acls-1562617375855/project-editors-455332057385',
 'kind': 'storage#bucketAccessControl',
 'projectTeam': {'projectNumber': '455332057385', 'team': 'editors'},
 'role': 'OWNER',
 'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855/acl/project-editors-455332057385'},
 {'bucket': 'bpo-acls-1562617375855',
 'entity': 'project-viewers-455332057385',
 'etag': 'CAI=',
 'id': 'bpo-acls-1562617375855/project-viewers-455332057385',
 'kind': 'storage#bucketAccessControl',
 'projectTeam': {'projectNumber': '455332057385', 'team': 'viewers'},
 'role': 'READER',
 'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855/acl/project-viewers-455332057385'}],
 'kind': 'storage#bucketAccessControls'}
(Pdb) client._connection.api_request(method="GET", path=self.bucket.path)
{'kind': 'storage#bucket', 'id': 'bpo-acls-1562617375855', 'selfLink': 'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617375855', 'projectNumber': '455332057385', 'name': 'bpo-acls-1562617375855', 'timeCreated': '2019-07-08T20:22:56.637Z', 'updated': '2019-07-08T20:22:58.175Z', 'metageneration': '2', 'iamConfiguration': {'bucketPolicyOnly': {'enabled': False}, 'uniformBucketLevelAccess': {'enabled': False}}, 'location': 'US', 'locationType': 'multi-region', 'storageClass': 'STANDARD', 'etag': 'CAI='}

Note that the resource's iamConfiguration contains 'bucketPolicyOnly': {'enabled': False} (?!)

[tseaver]

Under Python 2.7:

(Pdb) l
430 	 query_params = {}
431 	
432 	 if self.user_project is not None:
433 	 query_params["userProject"] = self.user_project
434 	
435 ->	 self.entities.clear()
436 	
437 	 found = client._connection.api_request(
438 	 method="GET", path=path, query_params=query_params
439 	 )
440 	 self.loaded = True
(Pdb) n
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(437)reload()
-> found = client._connection.api_request(
(Pdb) pp path
'/b/bpo-acls-1562617750458/acl'
(Pdb) pp query_params
{}
(Pdb) n
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(438)reload()
-> method="GET", path=path, query_params=query_params
(Pdb) 
BadRequest: BadReque...-only.',)
> /home/tseaver/projects/agendaless/Google/src/gcp/storage/google/cloud/storage/acl.py(438)reload()
-> method="GET", path=path, query_params=query_params
(Pdb) client._connection.api_request(method="GET", path=self.bucket.path)
{u'kind': u'storage#bucket', u'name': u'bpo-acls-1562617750458', u'timeCreated': u'2019-07-08T20:29:11.039Z', u'locationType': u'multi-region', u'updated': u'2019-07-08T20:29:12.575Z', u'projectNumber': u'455332057385', u'metageneration': u'2', u'location': u'US', u'iamConfiguration': {u'uniformBucketLevelAccess': {u'enabled': True, u'lockedTime': u'2019-10-06T20:29:12.539Z'}, u'bucketPolicyOnly': {u'enabled': True, u'lockedTime': u'2019-10-06T20:29:12.539Z'}}, u'etag': u'CAI=', u'id': u'bpo-acls-1562617750458', u'selfLink': u'https://www.googleapis.com/storage/v1/b/bpo-acls-1562617750458', u'storageClass': u'STANDARD'}

Here, the resource's iamConfiguration contains 'bucketPolicyOnly': {'enabled': True}, as expected.