Add "buildx" and "compose" CLI plugins (for supported architectures of each)

[tianon]

• ClosesDinD with docker-compose #303
• ClosesWhy doesn't docker:latest image contains the compose? #355
• ClosesMake buildx available in official Docker images docker/roadmap#322

(I don't actually have permission for this to auto-closedocker/roadmap#322, so maybe someone like @thaJeztah or @chris-crone can do the honors once we get this in.)

Refs:

• docker-compose #18
• Added a variant of Docker-in-Docker with docker compose #30
• docker-compose in docker #42
• Include docker-app plugin with docker image #156
• Include buildx cli-plugins #243
• buildx is missing from docker:20-dind images #313
• publish images for compose v2 to dockerhub docker/compose#8871
• [Docker Compose] V2 GA docker/roadmap#256
• [TAR-352] Add initial basis for cli-plugins (docker/app) docker/docker-ce-packaging#309

As a warning, this does very severely bloat the size of the Docker images (something like 100MiB more), but given these are part of the "Docker Experience" I think it makes sense. 🙈

[tao12345666333]

👍 great!

By the way, can we provide a tag like non-plugin, which means that plugin is not included.

Because in our actual scenario, DinD is heavily used in CI without the need to use plugins. It would be nice to not increase the size of the image. Thanks

[tianon]

🙈 that's exactly the type of request I've been hoping to avoid all this time 😅

If anything, I'd love to maintain less variations of this image than more. 😇

I've also been very strongly considering moving from Alpine to Debian or Ubuntu for the next (22.xx) release, since the experience of using Docker-Inc-provided Docker builds on both of those platforms is much more well-supported than Alpine is. 😞

[johnthagen]

I've also been very strongly considering moving from Alpine to Debian or Ubuntu for the next (22.xx) release

As a drive by comment, I too will say that I've hit rough edges working with the Alpine-derived Docker images. If you don't need to do any changes they work well, but the moment you want to add some scripts or anything else to the image, Alpine is harder to use.

Also big ❤️ for getting Compose v2 into an official image.

[johnthagen]

@tianon In the past, docker/compose had tags for each Docker Compose release: https://hub.docker.com/r/docker/compose/tags

How will Compose versioning work now (if, for example we want to pin to specific version of Compose?). Will the latest Compose simply be brought in at each new docker image release?

[tianon]

With the implementation currently in this PR, new versions of Compose will be integrated as they're released (so you'd have to pin to an image digest if you want something that doesn't change).

[wedi]

I am used to Alpine from the times when there was no great slim alternative and I have been sticking to it to copy and paste proven recipes. Now that we have, for example, an Ubuntu image with a footprint below 30 MB, I think that in principle it would be fine to switch. However, please keep in mind how many people use these images. Making the switch will easily require thousands of developer hours in downstream projects and Pipelines. So it would be nice if you kept it as long as it doesn't become a major drain on your time.

[wedi]

Once merged this would allow to finally resolve #156.

[tianon]

Just to record this somewhere, I did some testing on combining layers here to see if we get some compression efficiencies on transport (h/t @chris-crone), and the answer is a big resounding "no" -- building this as-is, the total "compressed size" of all layers is 89.47MB. If I combine both plugins layers into a single one, it's the same. Even if I combine the full Docker/containerd binaries layer as well, it's still exactly the same. 😬

(So my preference is to continue to leave them separate.)

[tianon]

I guess this is also a good place to note that the comparison to "current" from that is ~66.99MB, so for transport, this is not a huge increase.

[chris-crone]

Just to record this somewhere, I did some testing on combining layers here to see if we get some compression efficiencies on transport (h/t @chris-crone), and the answer is a big resounding "no" -- building this as-is, the total "compressed size" of all layers is 89.47MB. If I combine both plugins layers into a single one, it's the same. Even if I combine the full Docker/containerd binaries layer as well, it's still exactly the same. 😬

(So my preference is to continue to leave them separate.)

I was so sure that gzip would make the duplicated Go runtime go away but alas 😢

[johnthagen]

Now that Compose v2 is included in the docker image, should docker/compose (v1) be updated with a notice that it deprecated?

[thaJeztah]

Compose V2 is part of the CLI (through a plugin); compose v1 is a standalone binary only. As to "deprecated"; you can subscribe to docker/roadmap#257 for updates.

[johnthagen]

@tianon I don't know if it's moving up to Compose v2 or buildx or some combination, but when we moved our CI that runs docker/compose:1.29.2/docker-compose build --parallel -> docker/docker compose build our compose project (multiple multi-stage Dockerfiles) built twice as fast. ❤️

Thanks!