[BUG] Error response from daemon: invalid mode: rw,nocopy,www when using Volume subpath

[stivyw]

Description

I'm getting error when trying to use volume with subpath.
Error response from daemon: invalid mode: rw,nocopy,www when using Volume subpath

Steps To Reproduce

What does not work

volumes:
 myvol:
 driver: local
 driver_opts:
 o: bind
 type: none
 device: /sw/etc/balance

services:
 test:
 image: alpine
 command: sh -l -c 'ls /myvol && ls /var/myvol/* && ls /etc/myvol/*'
 volumes:
 - myvol:/myvol
 - type: volume
 source: myvol
 target: /var/myvol
 volume:
 # nocopy: true
 subpath: www
 - type: volume
 source: myvol
 target: /etc/myvol
 volume:
 nocopy: true
 subpath: etc
...

With local driver and opts, I continue with the same issue.

Error response from daemon: invalid mode: rw,nocopy,www

BUT When volume is external, it works:

mkdir -p $PWD/volume/etc $PWD/volume/www
touch $PWD/volume/etc/ETC $PWD/volume/www/WWW
docker volume create -d local -o type=none -o o=bind -o device=$PWD/volume testvol

docker-compose.yaml

volumes:
 myvol:
 name: testvol
 external: true
 # driver: local
 # driver_opts:
 # o: bind
 # type: none
 # device: $PWD/volume
services:
 test:
 image: alpine
 command: sh -l -c 'ls /myvol && ls /var/myvol/* && ls /etc/myvol/*'
 volumes:
 - myvol:/myvol
 - type: volume
 source: myvol
 target: /var/myvol
 volume:
 # nocopy: true
 subpath: www
 - type: volume
 source: myvol
 target: /etc/myvol
 volume:
 nocopy: true
 subpath: etc

Result

sws:/sw/apps/test$ docker compose up
[+] Running 2/1
 ✔ Network test_default Created 0.2s 
 ✔ Container test-test-1 Created 0.1s 
Attaching to test-1
test-1 | etc
test-1 | www
test-1 | /var/myvol/WWW
test-1 | /etc/myvol/ETC
test-1 exited with code 0
sws:/sw/apps/test$ 

So, I think maybe this is a docker-compose issue.

Compose Version

Docker Compose version v2.27.0
Docker Compose version v2.27.0

Docker Environment

Client:
 Version: 26.1.5
 Context: default
 Debug Mode: false
 Plugins:
 buildx: Docker Buildx (Docker Inc.)
 Version: v0.14.0
 Path: /usr/libexec/docker/cli-plugins/docker-buildx
 compose: Docker Compose (Docker Inc.)
 Version: v2.27.0
 Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 4
 Running: 3
 Paused: 0
 Stopped: 1
 Images: 9
 Server Version: 26.1.5
 Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Using metacopy: false
 Native Overlay Diff: true
 userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3a4de459a68952ffb703bbe7f2290861a75b6b67
 runc version: 51d5e94601ceffbbd85688df1c928ecccbfa4685
 init version: 
 Security Options:
 seccomp
 Profile: builtin
 cgroupns
 Kernel Version: 6.6.43-0-lts
 Operating System: Alpine Linux v3.20
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.664GiB
 Name: sws
 ID: bc9f87c4-621f-4201-b6a0-9e9faebeab52
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: true
 Insecure Registries:
 127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[ndeloof]

caused by:

• Fix bind mounts when in project volumes definition #9514 to force use of Bind API with volumes o: bind
• Add volume subpath option compose-spec/compose-go#594 to add subpath as a bind option while not supported by bind format

I'm investigating for a fix with risks for potential regressions in mind

[stivyw]

@ndeloof I think that was not fixed with #12078 .

[ndeloof]

#12078 has been reverted as change in API we use has too many impacts

[ndeloof]

the root cause is https://github.com/docker/compose/blob/main/pkg/compose/create.go#L1107-L1110
As compose is responsible to create such a volume, and detect use of local driver as a plain bind mount, it converts the volume definition accordingly (mount type=bind). But then, as we process binds to call the moby API https://github.com/docker/compose/blob/main/pkg/compose/create.go#L825-L826 we have to select use of the legacy bind API for backward compatibility, and only use the newer mount one for others. But as we converted a volume (with subpath) definition into a bind (which has no support for subpath) we just pass the raw bind string, which unfortunately includes configured subpath option, while being not supported.

IMHO we need to revisit 919f351 as considering volume definition (which includes new subpath options introduced by the mount API) is equivalent to a bind is wrong - those are not strictly equivalent by the way engine API is set.
cc @jhrotko

[stivyw]

I Understood. Thank's @ndeloof

[ndeloof]

fixed by https://github.com/docker/compose/pull/12177/files