migrate to moby modules by thaJeztah · Pull Request #13078 · docker/compose

thaJeztah · 2025-07-21T18:51:11Z

depends on pkg/compose: align classic builder implementation with docker/cli #13278
testing Create github.com/moby/moby/api and github.com/moby/moby/client module moby/moby#50280
testing migrate to moby/moby/api and moby/moby/client modules cli#6196
testing vendor: github.com/moby/moby/api v1.52.0, moby/client v0.1.0 buildx#3326

What I did

Related issue

(not mandatory) A picture of a cute animal, if possible in relation to what you did

ndeloof · 2025-07-21T18:56:05Z

Nice to see moby API as a dedicated module, so we don't get transitive dependencies from engine inside Compose ! ❤️

thaJeztah · 2025-07-21T18:58:32Z

@@ -0,0 +1,89 @@
+// Package urlutil provides helper function to check if a given build-context


Added this fork temporarily; didn't realise compose itself also uses it (not only through cli); the CLI fork is "internal", so probably needs to made public, or we keep a fork here 😞

thaJeztah · 2025-07-21T18:58:58Z

Remaining uses of docker/docker;

tree -d vendor/github.com/docker/docker
vendor/github.com/docker/docker
├── internal
│ └── lazyregexp
├── pkg
│ ├── jsonmessage
│ ├── namesgenerator
│ ├── pidfile
│ ├── process
│ ├── progress
│ ├── stdcopy
│ ├── streamformatter
│ └── stringid
└── registry
13 directories

thaJeztah · 2025-07-21T19:00:26Z

Build failure looks like it may be an incompatible change in buildx itself?

 > [build 1/1] RUN --mount=type=bind,target=. --mount=type=cache,target=/root/.cache --mount=type=cache,target=/go/pkg/mod --mount=type=bind,from=osxcross,src=/osxsdk,target=/xx-sdk xx-go --wrap && if [ "$(xx-info os)" == "darwin" ]; then export CGO_ENABLED=1; fi && make build GO_BUILDTAGS="e2e" DESTDIR=/out && xx-verify --static /out/docker-compose:
1.027 GO111MODULE=on go build -trimpath -tags "e2e" -ldflags "-w -X github.com/docker/compose/v2/internal.Version=fa50b46" -o "/out/docker-compose" ./cmd
38.90 # github.com/docker/buildx/build
38.90 /go/pkg/mod/github.com/tha!jeztah/buildx@v0.2.1-0.20250721183044-80b40a761230/build/provenance.go:135:16: pred1.ConvertToSLSA02 undefined (type *"github.com/moby/buildkit/solver/llbsolver/provenance/types".ProvenancePredicateSLSA1 has no field or method ConvertToSLSA02)
67.83 make: *** [Makefile:61: build] Error 1

38.90 /go/pkg/mod/github.com/tha!jeztah/buildx@v0.2.1-0.20250721183044-80b40a761230/build/provenance.go:135:16: pred1.ConvertToSLSA02 undefined (type

thaJeztah · 2025-07-21T19:10:48Z

OK, it's because buildx depends on an unreleased version of BuildKit, so go modules considers it to be older than the latest release and won't update; https://github.com/docker/buildx/blob/3f4bf829d8c5b92a8f670609417d9aa4c0b6be98/go.mod#L32

github.com/moby/buildkit v0.23.0-rc1.0.20250618182037-9b91d20367db // master

ndeloof · 2025-07-22T06:04:18Z

jsonmessage
this one should be part of the API, don't you think ?

thaJeztah · 2025-07-22T06:09:46Z

Yes, or at least the type; the utilities could be "client" - those packages are messy.

Also looking at things that are in the api, but shouldn't (api/types/plugins/logdriver)

thaJeztah · 2025-07-23T20:32:59Z

Also included a (WIP) branch for docker/cli#6202

Before/After;

Details

Before:

tree -d vendor/github.com/docker/cli
vendor/github.com/docker/cli
├── cli
│ ├── command
│ │ ├── completion
│ │ ├── container
│ │ ├── formatter
│ │ │ └── tabwriter
│ │ ├── image
│ │ │ └── build
│ │ │ └── internal
│ │ │ ├── git
│ │ │ └── urlutil
│ │ └── inspect
│ ├── compose
│ │ ├── interpolation
│ │ ├── loader
│ │ ├── schema
│ │ │ └── data
│ │ ├── template
│ │ └── types
│ ├── config
│ │ ├── configfile
│ │ ├── credentials
│ │ ├── memorystore
│ │ └── types
│ ├── connhelper
│ │ ├── commandconn
│ │ ├── internal
│ │ │ └── syntax
│ │ └── ssh
│ ├── context
│ │ ├── docker
│ │ └── store
│ ├── debug
│ ├── flags
│ ├── hints
│ ├── streams
│ ├── trust
│ └── version
├── cli-plugins
│ ├── hooks
│ ├── manager
│ ├── metadata
│ ├── plugin
│ └── socket
├── internal
│ ├── jsonstream
│ ├── lazyregexp
│ ├── prompt
│ └── tui
├── opts
│ └── swarmopts
├── pkg
│ └── kvfile
└── templates
55 directories

After:

tree -d vendor/github.com/docker/cli
vendor/github.com/docker/cli
├── cli
│ ├── command
│ │ ├── completion
│ │ ├── container
│ │ ├── formatter
│ │ │ └── tabwriter
│ │ ├── image
│ │ │ └── build
│ │ │ └── internal
│ │ │ ├── git
│ │ │ └── urlutil
│ │ └── inspect
│ ├── config
│ │ ├── configfile
│ │ ├── credentials
│ │ ├── memorystore
│ │ └── types
│ ├── connhelper
│ │ ├── commandconn
│ │ ├── internal
│ │ │ └── syntax
│ │ └── ssh
│ ├── context
│ │ ├── docker
│ │ └── store
│ ├── debug
│ ├── flags
│ ├── hints
│ ├── streams
│ ├── trust
│ └── version
├── cli-plugins
│ ├── hooks
│ ├── manager
│ ├── metadata
│ ├── plugin
│ └── socket
├── internal
│ ├── jsonstream
│ ├── lazyregexp
│ ├── prompt
│ ├── tui
│ └── volumespec
├── opts
├── pkg
│ └── kvfile
└── templates
48 directories

thaJeztah · 2025-08-05T22:52:03Z

Down to two packages; both possibly candidates for github.com/moby/sys;

tree -d vendor/github.com/docker/docker
vendor/github.com/docker/docker
└── pkg
├── pidfile
└── process
4 directories

CLI still needs sorting out;

Details

tree -d vendor/github.com/docker/cli
vendor/github.com/docker/cli
├── cli
│ ├── command
│ │ ├── completion
│ │ ├── container
│ │ ├── formatter
│ │ │ └── tabwriter
│ │ ├── image
│ │ │ └── build
│ │ │ └── internal
│ │ │ ├── git
│ │ │ └── urlutil
│ │ ├── inspect
│ │ └── system
│ │ └── pruner
│ ├── config
│ │ ├── configfile
│ │ ├── credentials
│ │ ├── memorystore
│ │ └── types
│ ├── connhelper
│ │ ├── commandconn
│ │ ├── internal
│ │ │ └── syntax
│ │ └── ssh
│ ├── context
│ │ ├── docker
│ │ └── store
│ ├── debug
│ ├── flags
│ ├── hints
│ ├── streams
│ ├── trust
│ └── version
├── cli-plugins
│ ├── hooks
│ ├── manager
│ ├── metadata
│ ├── plugin
│ └── socket
├── internal
│ ├── jsonstream
│ ├── lazyregexp
│ ├── prompt
│ ├── registry
│ ├── tui
│ └── volumespec
├── opts
├── pkg
│ └── kvfile
└── templates
51 directories

thaJeztah · 2026-01-22T10:21:20Z

-	// filter out useless commandConn.CloseWrite warning message that can occur
-	// when using a remote context that is unreachable: "commandConn.CloseWrite: commandconn: failed to wait: signal: killed"
-	// https://github.com/docker/cli/blob/e1f24d3c93df6752d3c27c8d61d18260f141310c/cli/connhelper/commandconn/commandconn.go#L203-L215
-	logrus.AddHook(logutil.NewFilter([]logrus.Level{


These were removed in docker/cli#6644

codecov · 2026-01-26T08:56:27Z

Codecov Report

❌ Patch coverage is 47.45989% with 393 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/dryrun/dryrunclient.go	0.00%	218 Missing ⚠️
pkg/compose/create.go	65.89%	37 Missing and 22 partials ⚠️
pkg/compose/push.go	0.00%	19 Missing ⚠️
cmd/compose/list.go	0.00%	15 Missing and 2 partials ⚠️
pkg/bridge/transformers.go	17.64%	13 Missing and 1 partial ⚠️
pkg/compose/cp.go	56.66%	13 Missing ⚠️
pkg/compose/generate.go	0.00%	13 Missing ⚠️
pkg/compose/pull.go	54.54%	8 Missing and 2 partials ⚠️
pkg/compose/events.go	0.00%	5 Missing ⚠️
pkg/compose/watch.go	80.00%	5 Missing ⚠️
... and 12 more

📢 Thoughts on this report? Let us know!

thaJeztah · 2026-02-10T15:49:17Z

@@ -18,7 +18,6 @@ require (
 	github.com/docker/cli v29.2.0+incompatible


Oh! Thought I had updated to v29.2.1; let me do so as well (no significant changes I think though)

Also update TestDefaultNetworkSettings: Test that the network with the highest priority is returned as "primary" network, and other networks as extra networks. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Format layer progress details with minimal efforts as new UI does not render individual layers Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Format layer progress details with minimal efforts as new UI does not render individual layers. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

glours

Sounds good to me, just 2 points we need to check

glours · 2026-02-10T16:45:28Z

 func (s *composeService) pruneDanglingImagesOnRebuild(ctx context.Context, projectName string, imageNameToIdMap map[string]string) {
-	images, err := s.apiClient().ImageList(ctx, image.ListOptions{
- Filters: filters.NewArgs(
- filters.Arg("dangling", "true"),


We lost the projectLabel filter, no?

Let me check!

Ah, right, so it now uses projectFilter;
images, err:=s.apiClient().ImageList(ctx, client.ImageListOptions{ Filters: projectFilter(projectName).Add("dangling", "true"), })
which should add it;
compose/pkg/compose/filters.go
Lines 27 to 29 in c75e494
funcprojectFilter(projectNamestring) client.Filters {
returnmake(client.Filters).Add("label", fmt.Sprintf("%s=%s", api.ProjectLabel, projectName))
}

glours · 2026-02-10T16:47:24Z

-	containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{
- All: true,
- Filters: filters.NewArgs(
- filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, project.Name)),


Same here as well, so I think we should be good 🫶

thaJeztah requested a review from a team as a code owner July 21, 2025 18:51

thaJeztah requested review from glours and ndeloof July 21, 2025 18:51

thaJeztah marked this pull request as draft July 21, 2025 18:51

thaJeztah changed the title ~~Bump moby cli~~WIP: migrate to moby modulesJul 21, 2025

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from a5b199b to 73bb946Compare July 21, 2025 18:55

thaJeztah commented Jul 21, 2025
View reviewed changes

Comment threadcmd/compose/bridge.go Outdated

thaJeztah commented Jul 21, 2025
View reviewed changes

thaJeztah mentioned this pull request Jul 21, 2025
Create github.com/moby/moby/api and github.com/moby/moby/client module moby/moby#50280
Merged

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from 83ca48f to 13827c4Compare July 21, 2025 19:20

thaJeztah force-pushed the bump_moby_cli branch from 13827c4 to 52f0ad8Compare July 23, 2025 20:32

thaJeztah force-pushed the bump_moby_cli branch 3 times, most recently from d2146d4 to 3a7124dCompare July 24, 2025 09:24

thaJeztah force-pushed the bump_moby_cli branch 3 times, most recently from 3b1ec4c to 6cd56afCompare August 5, 2025 22:51

thaJeztah force-pushed the bump_moby_cli branch from 6cd56af to 8925b9cCompare August 5, 2025 23:13

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from d7c329f to 04ac165Compare August 28, 2025 21:12

thaJeztah force-pushed the bump_moby_cli branch 7 times, most recently from a051b5a to 5098065Compare November 11, 2025 00:08

thaJeztah mentioned this pull request Nov 16, 2025
build(deps): bump buildx v0.30.0, buildkit v0.26.0, otel v1.38.0, otel/contrib v0.63.0 #13379
Merged

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from 70efab9 to 05f29edCompare November 16, 2025 14:11

thaJeztah changed the title ~~WIP: migrate to moby modules~~migrate to moby modulesNov 16, 2025

This was referenced Jan 15, 2026
build(deps): bump github.com/sirupsen/logrus v1.9.4 #13518
Merged
build(deps): bump github.com/moby/go-archive from 0.1.0 to 0.2.0 #13473
Closed

thaJeztah commented Jan 22, 2026
View reviewed changes

Comment threadpkg/compose/push.go Outdated

thaJeztah commented Jan 22, 2026
View reviewed changes

Comment threadpkg/compose/pull.go Outdated

thaJeztah commented Feb 4, 2026
View reviewed changes

Comment threadpkg/compose/create_test.go

ndeloof mentioned this pull request Feb 5, 2026
Format layer progress details with minimal efforts as new UI does not render individual layers thaJeztah/compose#335
Merged

thaJeztah mentioned this pull request Feb 10, 2026
pkg/compose: defaultNetworkSettings: refactor #13580
Merged
1 task

thaJeztah commented Feb 10, 2026
View reviewed changes

thaJeztahand others added 5 commits February 10, 2026 16:51

go.mod: bump github.com/moby/moby/api v1.53.0, moby/client v0.2.2
c75e494
Also update TestDefaultNetworkSettings: Test that the network with the highest priority is returned as "primary" network, and other networks as extra networks. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/bridge: remove uses of go-connections
f168567
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/compose: remove unused consts
eb7130e
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/compose: format layer pull progress
e001ebe
Format layer progress details with minimal efforts as new UI does not render individual layers Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/compose: format layer push progress
47a2724
Format layer progress details with minimal efforts as new UI does not render individual layers. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

glours approved these changes Feb 10, 2026
View reviewed changes

vvoland approved these changes Feb 10, 2026
View reviewed changes

BrewTestBot mentioned this pull request Feb 24, 2026
docker-compose 5.1.0 Homebrew/homebrew-core#269211
Merged

rancherbot mentioned this pull request Feb 25, 2026
rddepman: bump dockerCompose from 5.0.2 to 5.1.0 rancher-sandbox/rancher-desktop#9931
Merged

		@@ -0,0 +1,89 @@
		// Package urlutil provides helper function to check if a given build-context

		@@ -18,7 +18,6 @@ require (
		github.com/docker/cli v29.2.0+incompatible

	funcprojectFilter(projectNamestring) client.Filters {
	returnmake(client.Filters).Add("label", fmt.Sprintf("%s=%s", api.ProjectLabel, projectName))
	}

Uh oh!

Conversation

thaJeztah commented Jul 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ndeloof commented Jul 21, 2025

Uh oh!

Uh oh!

thaJeztahJul 21, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

ndeloof commented Jul 22, 2025

Uh oh!

thaJeztah commented Jul 22, 2025

Uh oh!

thaJeztah commented Jul 23, 2025

Uh oh!

thaJeztah commented Aug 5, 2025

Uh oh!

thaJeztahJan 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecovBot commented Jan 26, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

thaJeztahFeb 10, 2026

Choose a reason for hiding this comment

Uh oh!

glours left a comment

Choose a reason for hiding this comment

Uh oh!

gloursFeb 10, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztahFeb 10, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztahFeb 10, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gloursFeb 10, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztahFeb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thaJeztah commented Jul 21, 2025•
edited
Loading

codecovBot commented Jan 26, 2026•
edited
Loading

thaJeztahFeb 10, 2026•
edited
Loading