Skip to content

Add support for regenerating protocol buffers, and refresh .pb.go files - #2193

Merged
ctelfer merged 2 commits into
moby:masterfrom
euanh:update-protobuf
Jun 27, 2018
Merged

Add support for regenerating protocol buffers, and refresh .pb.go files#2193
ctelfer merged 2 commits into
moby:masterfrom
euanh:update-protobuf

Conversation

@euanh

@euanheuanh commented Jun 20, 2018

Copy link
Copy Markdown
Contributor

Previously the build container did not provide protoc or protoc-gen-gogo; now these are available in the container. There is also a make pattern rule to regenerate .pb.go files from .proto files.

I regenerated the .pb.go files. agent.go.pb was regenerated recently and does not change, however older .pb.go files change slightly when regenerated with newer tools.

@euanh

euanh commented Jun 20, 2018

Copy link
Copy Markdown
ContributorAuthor

@fcrisciani, @ctelfer: This should make regenerating protocol buffers more consistent. I'll try to come up with a nice regenerate-protobufs target in the Makefile as well. In the meantime, just touch the .proto file or delete the .pb.go and run make <path/foo>.pb.go from within the build container.

@ctelferctelfer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely a worthwhile thing to add and the output looks sane from a brief glance. Have you tried building moby with the resulting libnetwork. Best double-check would be to replace one of the moby's in a cluster with one w/ the updated libnetwork and then run some test load of services to make sure all nodes can still communicate cleanly w/ each other. (I can't imagine the PB mods will break ... but need to verify of course.)

@euanh

euanh commented Jun 21, 2018

Copy link
Copy Markdown
ContributorAuthor

I haven't tried building moby with these changes yet. Is there a convenient set of tests which particularly exercises libnetwork?

@euanh
euanhforce-pushed the update-protobuf branch 2 times, most recently from 1d15e95 to ebc2fdaCompareJune 22, 2018 13:15
euanh added 2 commits June 22, 2018 15:03
Signed-off-by: Euan Harris <euan.harris@docker.com>
agent.pb.go is unchanged, but the files in networkdb and drivers
are slightly different when regenerated using the current versions
of protoc and gogoproto. This is probably because agent.pb.go
was last regenerated quite recently, in February 2018, whereas
networkdb.pb.go and overlay/overlay.pb.go were last changed in 2017,
and windows/overlay/overlay.pb.go was last changed in 2016.
Signed-off-by: Euan Harris <euan.harris@docker.com>
@euanheuanh changed the title WIP: Add support for regenerating protocol buffers, and refresh .pb.go filesAdd support for regenerating protocol buffers, and refresh .pb.go filesJun 22, 2018
@euanh

Copy link
Copy Markdown
ContributorAuthor

I tried using the vendored version of gogoproto, but protoc generated bad import statements which go build couldn't resolve:

import _ "gogoproto"

instead of

import _ "github.com/gogo/protobuf/gogoproto"

@fcriscianifcrisciani left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment threadMakefile
PB_FILES=$(PROTO_FILES:.proto=.pb.go)

%.pb.go: %.proto
${docker} protoc -I=. -I=/go/src -I=/go/src/github.com/gogo/protobuf -I=/go/src/github.com/gogo/protobuf/protobuf --gogo_out=./ $<

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't not have to be in this PR, but would be great to fail if the PR while compiled by the CI has a diff with the file generated compared to the files coming from the PR itself. This way we can enforce that the PR contains all the proper information.
Also does it makes sense to have build target depend on the protobuf generation?

@selansen

Copy link
Copy Markdown
Contributor

LGTM

@euanh

Copy link
Copy Markdown
ContributorAuthor

@fcrisciani@selansen@ctelfer I'm currently running the tests that @ctelfer suggested and tracking down a couple of failures.

@euanh

Copy link
Copy Markdown
ContributorAuthor

I reproduced the same failures running a mixed-version cluster with moby master (without my changes) on the leader and 17.06 on two other nodes. Updating all nodes to moby master made them pass. Similarly, with my changes only on the leader I had a couple of failures; with my changes on all nodes the tests passed.

@ctelferctelfer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@euanh@selansen@fcrisciani@ctelfer