chore: migrate to owl bot

[parthea]

This PR migrates from autosynth to owl bot. owl bot will save time for maintainers as it will automatically open PRs when there are updates in googleapis-gen without requiring maintainers to run synthtool to build the client from protos. Additionally, similar to autosynth, PRs will be automatically opened when there are template updates.

[pradn]

Is it possible to test this before submitting?

[plamut]

@pradn I had the exact same question. The docs are in the making, but until then, the steps posted here should be sufficient to run the bot locally and test how it performs.

I'll try it out myself too early next week.

[pradn]

Docs to learn more about this:
https://g3doc.corp.google.com/company/teams/dpe-github-automation/owlbot.md?cl=head

[plamut]

I don't have access to Google internals. :)

I was not able to pull an image from gcr.io/repo-automation-bots to test this PR. I probably lack permissions on repo-automation-bots, as I can pull some other images from gcr.io just fine (namely, those that I uploaded myself to the common development Google Cloud project).

[parthea]

The OwlBot Post Processor is already included in the checks section as a result of this PR. It is one of the 9 successful checks. The docker file used to create the image lives here. For testing purposes, you could run gcloud builds submit --config=docker/owlbot/python/cloudbuild.yaml --substitutions=SHORT_SHA=$(git rev-parse --short HEAD) from the root of https://github.com/googleapis/synthtool to deploy your own post processor image. You would need to specify a PROJECT_ID which is used here.

[plamut]

@parthea Thanks, that was helpful! Missed that new CI check, as it was hidden (the checklist is collapsed by default).

I successfully built and uploaded the image to "our" development Google Cloud project, pulled it, and ran it locally - it worked like a charm.

I also verified that a replacement rule change in owlbot.py was indeed reflected in the generated code.

Looking forward to the docs update to explain how to run owlbot locally when developing, but as far as this PR is concerned, I think it's good.

[plamut]

One minor thing (not a blocker) - the CI check say it successfully ran 9 steps, but without any details. What would be the best way to diagnose any errors if that check fails?

(possible example: a bad replacement rule in owlbot.py)

[plamut]

@pradn Do you have anything to add here?

[plamut]

Let's just wait with merging until the release PR is done.

[parthea]

Let's wait for googleapis/repo-automation-bots#1790 to see if that helps to view logs from owl bot.

[plamut]

Release completed (last week already).

Still not sure if we should merge this, though, because it would be quite impractical for external contributors to build their own Docker images and upload them to where they can actually access them.

[parthea]

@plamut Sounds good. Let's wait for googleapis/repo-automation-bots#1790

[plamut]

For the record - after #299 is merged, we need to port its changes to synth.py to owlbot.py here.

[plamut]

@parthea Do we have an update on this?

I noticed in #451 that the library suffers from the gpg: keyserver receive failed error, but did not benefit from the fix like BigQuery did, because owlbot migration is not yet done here.

Please advise, thanks!

[parthea]

Hi @plamut ,

I've pushed a new image to cloud-devrel-public-resources. Please can you try running docker pull gcr.io/cloud-devrel-public-resources/owlbot-python:latest as a test to see if you have access?

[plamut]

@parthea Ran out of time today for Pub/Sub, but will try it net week.

[parthea]

I ran these commands to copy the generated code for pubsub from googleapis-gen.

docker pull gcr.io/repo-automation-bots/owlbot-cli:latest

docker run --rm --user $(id -u):$(id -g) -v $(pwd):/repo -w /repo gcr.io/repo-automation-bots/owlbot-cli:latest copy-code

docker pull gcr.io/cloud-devrel-public-resources/owlbot-python:latest

docker run --user $(id -u):$(id -g) --rm -v $(pwd):/repo -w /repo gcr.io/cloud-devrel-public-resources/owlbot-python:latest

You should now be able to run the last 2 commands. The owlbot-cli image is not available in the cloud-devrel-public-resources project yet. I've added an item to the agenda for the github automation team meeting to discuss whether it is possible to copy the owlbot-cli image to cloud-devrel-public-resources.

[parthea]

Hi @plamut, @pradn,

My initial thought is that we should go ahead and merge this without waiting for the owlbot-cli to be available in cloud-devrel-public-resources. It is not common to run the owlbot-cli manually as owlbot does that automatically for you once the migration is complete. This is the last python repo in googleapis that is still using autosynth. This PR is needed to complete the Yoshi Python migration to owl bot. You can now run the post processor image locally using these commands. WDYT?

docker pull gcr.io/cloud-devrel-public-resources/owlbot-python:latest

docker run --user $(id -u):$(id -g) --rm -v $(pwd):/repo -w /repo gcr.io/cloud-devrel-public-resources/owlbot-python:latest

[plamut]

@parthea If we can count on support in uncommon cases when manual intervention might be required, I'd say let's move things forward and complete the transition. Especially because keeping this open is prone to additional merge conflicts, the process requiring special handling of this library, and other overhead.

I'll wait for @pradn to also have a look.

[pradn]

Looks good, thanks!

[plamut]

FWIW, I tried running the two commands from above and it worked like a charm. I successfully generated new changes locally (a new Samples section in CONTRIBUTING.rst in this particular case).