scripts/build: make sure osusergo is set for static with CGO enabled by thaJeztah · Pull Request #3490 · docker/cli

thaJeztah · 2022-03-26T14:50:27Z

Our previous CI probably did not cross-compile for s390x and ppc64le,
and therefore did not get these errors; the scripts/build/.variables
file sets CGO_ENABLED=1 for amd64|arm64|arm|s390x architectures if
it's not set;

cli/scripts/build/.variables

Lines 34 to 46 in 87b8e57

 linux) 
case"$(go env GOARCH)"in
 amd64|arm64|arm|s390x) 
 CGO_ENABLED=1 
 ;; 
*) 
 CGO_ENABLED=0 
 ;; 
esac
 ;; 
 darwin|windows) 
 CGO_ENABLED=1 
 ;; 

When compiling statically with CGO enabled, we must have osusergo
enabled as well, so set it accordingly, to prevent;

#53 [linux/amd64 build 2/2] RUN --mount=type=bind,target=.,ro --mount=type=cache,target=/root/.cache --mount=from=dockercore/golang-cross:xx-sdk-extras,target=/xx-sdk,src=/xx-sdk --mount=type=tmpfs,target=cli/winresources xx-go --wrap && TARGET=/out ./scripts/build/binary && xx-verify $([ "static" = "static" ] && echo "--static") /out/docker
#53 953.6 # github.com/docker/cli/cmd/docker
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000022.o: in function `New':
#53 953.6 /go/src/github.com/docker/cli/vendor/github.com/miekg/pkcs11/pkcs11.go:75: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000018.o: in function `mygetgrouplist':
#53 953.6 /usr/local/go/src/os/user/getgrouplist_unix.go:18: warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetgrgid_r':
#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:40: warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetgrnam_r':
#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:45: warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetpwnam_r':
#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:35: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetpwuid_r':
#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:30: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000004.o: in function `_cgo_3c1cec0c9a4e_C2func_getaddrinfo':
#53 953.6 /tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Our previous CI probably did not cross-compile for s390x and ppc64le, and therefore did not get these errors; the scripts/build/.variables file sets CGO_ENABLED=1 for amd64|arm64|arm|s390x architectures if it's not set; https://github.com/docker/cli/blob/87b8e57868adba9e189e9fb74e5bd5d189a024c8/scripts/build/.variables#L34-L46 When compiling statically with CGO enabled, we must have osusergo enabled as well, so set it accordingly, to prevent; docker#53 [linux/amd64 build 2/2] RUN --mount=type=bind,target=.,ro --mount=type=cache,target=/root/.cache --mount=from=dockercore/golang-cross:xx-sdk-extras,target=/xx-sdk,src=/xx-sdk --mount=type=tmpfs,target=cli/winresources xx-go --wrap && TARGET=/out ./scripts/build/binary && xx-verify $([ "static" = "static" ] && echo "--static") /out/docker docker#53 953.6 # github.com/docker/cli/cmd/docker docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000022.o: in function `New': docker#53 953.6 /go/src/github.com/docker/cli/vendor/github.com/miekg/pkcs11/pkcs11.go:75: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000018.o: in function `mygetgrouplist': docker#53 953.6 /usr/local/go/src/os/user/getgrouplist_unix.go:18: warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetgrgid_r': docker#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:40: warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetgrnam_r': docker#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:45: warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetpwnam_r': docker#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:35: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000017.o: in function `mygetpwuid_r': docker#53 953.6 /usr/local/go/src/os/user/cgo_lookup_unix.go:30: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#53 953.6 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-2402600021/000004.o: in function `_cgo_3c1cec0c9a4e_C2func_getaddrinfo': docker#53 953.6 /tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah · 2022-03-26T14:51:08Z

splitting up #3383 (where we're debugging some issues with cross compiles); moving this to a separate PR so that we can test/merge some changes in isolation.

codecov-commenter · 2022-03-26T14:52:22Z

Codecov Report

Merging #3490 (f894016) into master (87b8e57) will not change coverage.
The diff coverage is n/a.

❗ Current head f894016 differs from pull request most recent head e20572d. Consider uploading reports for the commit e20572d to get more accurate results

@@ Coverage Diff @@## master #3490 +/- ##
=======================================
Coverage 59.01% 59.01% =======================================
Files 284 284 Lines 23833 23833 =======================================
Hits 14066 14066 Misses 8908 8908 Partials 859 859

thaJeztah · 2022-03-26T14:52:34Z

FWIW; I think this is only an issue when using glibc for static compiles (but not sure if we need to add more complexity to handle this for other cases)

thaJeztah · 2022-03-26T15:05:46Z

Hmm... looks like we're still missing it somewhere; less errors, but still some;

#32 [linux/amd64->s390x build 2/2] RUN --mount=type=bind,target=.,ro --mount=type=cache,target=/root/.cache --mount=from=dockercore/golang-cross:xx-sdk-extras,target=/xx-sdk,src=/xx-sdk --mount=type=tmpfs,target=cli/winresources xx-go --wrap && TARGET=/out ./scripts/build/binary && xx-verify $([ "static" = "static" ] && echo "--static") /out/docker
#32 467.0 # github.com/docker/cli/cmd/docker
#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000018.o: in function `New':
#32 467.0 /go/src/github.com/docker/cli/vendor/github.com/miekg/pkcs11/pkcs11.go:75: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000004.o: in function `_cgo_26061493d47f_C2func_getaddrinfo':
#32 467.0 /tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

thaJeztah · 2022-03-26T15:21:40Z

Ah, never mind that's not the osusergo problem, but an issue with pkcs11, effectively means we cannot compile statically with pkcs11 enabled if we have CGO_ENABLED=1 and use glibc;

cli/vendor/github.com/miekg/pkcs11/pkcs11.go

Line 75 in 3897c3f

c->handle = dlopen(module, RTLD_LAZY);

pkcs11 cannot be compiled statically if CGO is enabled (and glibc is used) see https://github.com/docker/cli/blob/3897c3fa544f1239c2bc2c3af2adcebcb3650c4d/vendor/github.com/miekg/pkcs11/pkcs11.go#L75 see docker#3490 (comment) When trying to do otherwise, warnings are printed (and shouldn't be ignored, similar to the `osusergo` build-tag): docker#32 [linux/amd64->s390x build 2/2] RUN --mount=type=bind,target=.,ro --mount=type=cache,target=/root/.cache --mount=from=dockercore/golang-cross:xx-sdk-extras,target=/xx-sdk,src=/xx-sdk --mount=type=tmpfs,target=cli/winresources xx-go --wrap && TARGET=/out ./scripts/build/binary && xx-verify $([ "static" = "static" ] && echo "--static") /out/docker docker#32 467.0 # github.com/docker/cli/cmd/docker docker#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000018.o: in function `New': docker#32 467.0 /go/src/github.com/docker/cli/vendor/github.com/miekg/pkcs11/pkcs11.go:75: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000004.o: in function `_cgo_26061493d47f_C2func_getaddrinfo': docker#32 467.0 /tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah · 2022-03-26T15:32:28Z

OK, static "check" is failing; Possibly this is due to how musl works; similar to the discussion on docker/docker-ce-packaging#430

#31 0.295 + go build -o /out/docker-linux-amd64 -tags ' osusergo' -ldflags ' -w -X "github.com/docker/cli/cli/version.GitCommit=f3237fc" -X "github.com/docker/cli/cli/version.BuildTime=2022-03-26T15:27:37Z" -X "github.com/docker/cli/cli/version.Version=20.10.0-dev" -extldflags -static' '-buildmode=pie' github.com/docker/cli/cmd/docker
#31 67.22 file /out/docker is not statically linked: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, Go BuildID=qUFf96xsGR5l7Qlv9vEZ/g3YM480caVLNyoRtCyDU/75GBb7i6f1JkVSTP2aFV/aePmzjOgxVpdjhtqKKPL, not stripped

/out/docker is not statically linked: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1

thaJeztah · 2022-03-26T15:33:23Z

OK; let's move that last fix separate

pkcs11 cannot be compiled statically if CGO is enabled (and glibc is used) see https://github.com/docker/cli/blob/3897c3fa544f1239c2bc2c3af2adcebcb3650c4d/vendor/github.com/miekg/pkcs11/pkcs11.go#L75 see docker#3490 (comment) When trying to do otherwise, warnings are printed (and shouldn't be ignored, similar to the `osusergo` build-tag): docker#32 [linux/amd64->s390x build 2/2] RUN --mount=type=bind,target=.,ro --mount=type=cache,target=/root/.cache --mount=from=dockercore/golang-cross:xx-sdk-extras,target=/xx-sdk,src=/xx-sdk --mount=type=tmpfs,target=cli/winresources xx-go --wrap && TARGET=/out ./scripts/build/binary && xx-verify $([ "static" = "static" ] && echo "--static") /out/docker docker#32 467.0 # github.com/docker/cli/cmd/docker docker#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000018.o: in function `New': docker#32 467.0 /go/src/github.com/docker/cli/vendor/github.com/miekg/pkcs11/pkcs11.go:75: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000004.o: in function `_cgo_26061493d47f_C2func_getaddrinfo': docker#32 467.0 /tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah · 2022-03-26T17:27:46Z

opened #3494 for the pkcs11 issue

pkcs11 cannot be compiled statically if CGO is enabled (and glibc is used) see https://github.com/docker/cli/blob/3897c3fa544f1239c2bc2c3af2adcebcb3650c4d/vendor/github.com/miekg/pkcs11/pkcs11.go#L75 see docker#3490 (comment) When trying to do otherwise, warnings are printed (and shouldn't be ignored, similar to the `osusergo` build-tag): docker#32 [linux/amd64->s390x build 2/2] RUN --mount=type=bind,target=.,ro --mount=type=cache,target=/root/.cache --mount=from=dockercore/golang-cross:xx-sdk-extras,target=/xx-sdk,src=/xx-sdk --mount=type=tmpfs,target=cli/winresources xx-go --wrap && TARGET=/out ./scripts/build/binary && xx-verify $([ "static" = "static" ] && echo "--static") /out/docker docker#32 467.0 # github.com/docker/cli/cmd/docker docker#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000018.o: in function `New': docker#32 467.0 /go/src/github.com/docker/cli/vendor/github.com/miekg/pkcs11/pkcs11.go:75: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking docker#32 467.0 /usr/bin/s390x-linux-gnu-ld: /tmp/go-link-900380651/000004.o: in function `_cgo_26061493d47f_C2func_getaddrinfo': docker#32 467.0 /tmp/go-build/cgo-gcc-prolog:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah added status/2-code-review process/cherry-pick kind/bugfix PR's that fix bugs labels Mar 26, 2022

thaJeztah added this to the 21.xx milestone Mar 26, 2022

thaJeztah changed the title ~~scripts/build: make sure osusergo is set for static with CGO enabled~~scripts/build: make sure osusergo is set, and pkcs11 unset for static with CGO enabledMar 26, 2022

thaJeztah force-pushed the fix_osusergo branch from f894016 to e20572dCompare March 26, 2022 15:33

thaJeztah changed the title ~~scripts/build: make sure osusergo is set, and pkcs11 unset for static with CGO enabled~~scripts/build: make sure osusergo is set for static with CGO enabledMar 26, 2022

crazy-max approved these changes Mar 26, 2022
View reviewed changes

thaJeztah merged commit 0b266ba into docker:masterMar 26, 2022

thaJeztah deleted the fix_osusergo branch March 26, 2022 16:00

thaJeztah mentioned this pull request Mar 26, 2022
scripts/build/binary: disable pkcs11 on static builds with CGO_ENABLED=1 #3494
Closed

thaJeztah removed the process/cherry-pick label Feb 15, 2023

Navigation Menu

Uh oh!

scripts/build: make sure osusergo is set for static with CGO enabled - #3490

scripts/build: make sure osusergo is set for static with CGO enabled#3490
thaJeztah merged 1 commit into
docker:masterfrom
thaJeztah:fix_osusergo

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

codecov-commenter commented Mar 26, 2022•
edited
Loading

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	linux)
	case"$(go env GOARCH)"in
	amd64\|arm64\|arm\|s390x)
	CGO_ENABLED=1
	;;
	*)
	CGO_ENABLED=0
	;;
	esac
	;;
	darwin\|windows)
	CGO_ENABLED=1
	;;

Uh oh!

Conversation

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

codecov-commenter commented Mar 26, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

thaJeztah commented Mar 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Mar 26, 2022•
edited
Loading