Add system tests for bucket notifications. by tseaver · Pull Request #4021 · googleapis/google-cloud-python

tseaver · 2017-09-21T21:30:13Z

N.B.: the tests currently fail with a 403:

____________TestStorageNotificationCRUD.test_notification_explicit____________Traceback (mostrecentcalllast):
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/tests/system.py", line628, intest_notification_explicitretry_429(notification.create)()
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/.nox/sys-3-6/lib/python3.6/site-packages/test_utils/retry.py", line95, inwrapped_functionreturnto_wrap(*args, **kwargs)
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/google/cloud/storage/notification.py", line243, increatedata=properties,
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/.nox/sys-3-6/lib/python3.6/site-packages/google/cloud/_http.py", line293, inapi_requestraiseexceptions.from_http_response(response)
google.api.core.exceptions.Forbidden: 403POSThttps://www.googleapis.com/storage/v1/b/notification-explicit-1506029348265/notificationConfigs: Theserviceaccount'precise-truck-742@gs-project-accounts.iam.gserviceaccount.com'doesnothavepermissiontopublishmessagestototheCloudPub/Subtopic'//pubsub.googleapis.com/projects/precise-truck-742/topics/notification-1506029282321', orthattopicdoesnotexist.
____________TestStorageNotificationCRUD.test_notification_minimal_____________Traceback (mostrecentcalllast):
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/tests/system.py", line607, intest_notification_minimalretry_429(notification.create)()
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/.nox/sys-3-6/lib/python3.6/site-packages/test_utils/retry.py", line95, inwrapped_functionreturnto_wrap(*args, **kwargs)
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/google/cloud/storage/notification.py", line243, increatedata=properties,
File"/home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage/.nox/sys-3-6/lib/python3.6/site-packages/google/cloud/_http.py", line293, inapi_requestraiseexceptions.from_http_response(response)
google.api.core.exceptions.Forbidden: 403POSThttps://www.googleapis.com/storage/v1/b/notification-minimal-1506029351126/notificationConfigs: Theserviceaccount'precise-truck-742@gs-project-accounts.iam.gserviceaccount.com'doesnothavepermissiontopublishmessagestototheCloudPub/Subtopic'//pubsub.googleapis.com/projects/precise-truck-742/topics/notification-1506029282321', orthattopicdoesnotexist.

See ~~#3956~~https://github.com/GoogleCloudPlatform/google-cloud-common/issues/231 where I ask for help on that issue.

dhermes

No real complaints here

Sign in to view

 session.install('../test_utils/')
- session.install('.')
+ session.install('../pubsub')
+ session.install('-e', '.')


Sign in to view

+ with Config.CLIENT.batch():
+ for bucket_name in self.case_buckets_to_delete:
+ bucket = Config.CLIENT.bucket(bucket_name)
+ retry_429(bucket.delete)()


Sign in to view

+ self.assertEqual(len(notifications), 1)
+ self.assertEqual(notifications[0].topic_name, self.TOPIC_NAME)
+ finally:
+ notification.delete()


Sign in to view

+ self.assertIsNotNone(notification.id)
+ notifications = list(bucket.list_notifications())
+ self.assertEqual(len(notifications), 1)
+ self.assertEqual(notifications[0].topic_name, self.TOPIC_NAME)


tseaver · 2017-09-22T17:03:32Z

7d9ee94 updates the notification system tests, adding the IAM grant to the topic required to allow creation of the notification. Those tests are now passing (61c5d0f and da6dc1c fix bugs uncovered by running them).

tseaver · 2017-09-22T19:08:09Z

Branch build with new system tests passes the storage system tests. CircleCI failures are unrelated vision flakiness (#4035)

Default it to 'NONE'.

N.B.: the tests currently fail with a 403.

The back-end uses 'object_name_prefix'.

tseaver · 2017-09-25T15:34:36Z

I rebased the target branch against master after the merge of #4050, and rebased this PR's branch against that branch: here's hoping the "dandruff shampoo" worked.

Sign in to view

+ policy = self.publisher_client.get_iam_policy(self.topic_path)
+ binding = policy.bindings.add()
+ binding.role = 'roles/pubsub.publisher'
+ binding.members.append(


tseaver added api: storage Issues related to the Cloud Storage API. do not merge Indicates a pull request not ready for merge, due to either quality or timing. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Sep 21, 2017

tseaver requested a review from lukesneeringer as a code owner September 21, 2017 21:30

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Sep 21, 2017

dhermes reviewed Sep 21, 2017
View reviewed changes

tseaver changed the title ~~WIP: Add system tests for bucket notifications.~~Add system tests for bucket notifications.Sep 22, 2017

tseaver removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Sep 22, 2017

tseaver mentioned this pull request Sep 22, 2017
Add support for storage->pubsub notifications #3956
Closed
8 tasks

tseaver force-pushed the 3956-storage-notifications branch from 9eb9a46 to 352aeedCompare September 25, 2017 15:32

tseaver added 6 commits September 25, 2017 11:32

'payload_format' cannot be omitted.
20abf45
Default it to 'NONE'.

Install in develop mode.
8add165

WIP: Add system tests for bucket notifications.
d303d5e
N.B.: the tests currently fail with a 403.

Fix on-the-wire repr for 'blob_name_prefix'.
5247a28
The back-end uses 'object_name_prefix'.

Typo fix.
82dca43

Add required IAM for notification topic.
0fed445

BrandonY reviewed Sep 27, 2017
View reviewed changes

Comment threadstorage/tests/system.py
policy = self.publisher_client.get_iam_policy(self.topic_path)
binding = policy.bindings.add()
binding.role = 'roles/pubsub.publisher'
binding.members.append(
This comment was marked as spam.
Sign in to view

tseaver mentioned this pull request Oct 2, 2017
GCS notification convenience methods, pubsub dependency #4097
Closed

tseaver merged commit 978eb0f into googleapis:3956-storage-notificationsOct 5, 2017

tseaver deleted the 3956-storage-notifications-systest branch October 5, 2017 17:34

Uh oh!

Conversation

tseaver commented Sep 21, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dhermes left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

tseaver commented Sep 22, 2017

Uh oh!

tseaver commented Sep 22, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tseaver commented Sep 25, 2017

Uh oh!

This comment was marked as spam.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tseaver commented Sep 21, 2017•
edited
Loading

tseaver commented Sep 22, 2017•
edited
Loading