Empty string "--volume" causes unhelpful error messages

[dries007]

Description

When you start a docker container with an empty string passed to --volume you get unhelpful error messages.

I would like to see that this is caught by the CLI with a nice error message.

Reproduce

docker run -v "" --rm hello-world

Output:

docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory: unknown.

Expected behavior

Something along the lines of invalid argument "" for "--volume" flag: EOF.

This would match what happens when you run docker run --mount "" --rm hello-world.

docker version

Client:
 Version: 23.0.1
 API version: 1.42
 Go version: go1.20
 Git commit: a5ee5b1dfc
 Built: Sat Feb 11 13:58:04 2023
 OS/Arch: linux/amd64
 Context: default

Server:
 Engine:
 Version: 23.0.1
 API version: 1.42 (minimum version 1.12)
 Go version: go1.20
 Git commit: bc3805a0a0
 Built: Sat Feb 11 13:58:04 2023
 OS/Arch: linux/amd64
 Experimental: false
 containerd:
 Version: v1.6.18
 GitCommit: 2456e983eb9e37e47538f59ea18f2043c9a73640.m
 runc:
 Version: 1.1.4
 GitCommit: 
 docker-init:
 Version: 0.19.0
 GitCommit: de40ad0

docker info

Client:
 Context: default
 Debug Mode: false
 Plugins:
 buildx: Docker Buildx (Docker Inc.)
 Version: 0.10.3
 Path: /usr/lib/docker/cli-plugins/docker-buildx
 compose: Docker Compose (Docker Inc.)
 Version: 2.16.0
 Path: /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 31
 Running: 2
 Paused: 0
 Stopped: 29
 Images: 1099
 Server Version: 23.0.1
 Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Using metacopy: true
 Native Overlay Diff: false
 userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640.m
 runc version: 
 init version: de40ad0
 Security Options:
 seccomp
 Profile: builtin
 cgroupns
 Kernel Version: 6.1.12-zen1-1-zen
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 31.29GiB
 Name: DriesPC
 ID: 3S7A:XOUF:QR4M:SBSF:25EG:PTGE:UDJP:H7NB:PQB6:W473:DHCH:QEH7
 Docker Root Dir: /mnt/ssd2/docker
 Debug Mode: false
 Username: dries007
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
 127.0.0.0/8
 Live Restore Enabled: false

Additional Info

This happened because I was using a script in which a variable was accidentally unset. This caused a very long search since it was not obvious what was the root cause.

[thaJeztah]

Thanks for reporting! Yes, looks like there's some improvements to be made 🤔

I gave your example a try both on Docker 23.0.1 and 20.10, and it looks like the same happens on both, so the "good new" is that it's not a regression.

When putting the daemon in "debug mode", I can see that the following API calls are made;

DEBU[2023-02-27T16:08:26.716504702Z] Calling POST /v1.42/containers/create
DEBU[2023-02-27T16:08:26.718090142Z] form data: {"AttachStderr":true,"AttachStdin":false,"AttachStdout":true,"Cmd":null,"Domainname":"","Entrypoint":null,"Env":null,"HostConfig":{"AutoRemove":true,"Binds":null,"BlkioDeviceReadBps":[],"BlkioDeviceReadIOps":[],"BlkioDeviceWriteBps":[],"BlkioDeviceWriteIOps":[],"BlkioWeight":0,"BlkioWeightDevice":[],"CapAdd":null,"CapDrop":null,"Cgroup":"","CgroupParent":"","CgroupnsMode":"","ConsoleSize":[60,234],"ContainerIDFile":"","CpuCount":0,"CpuPercent":0,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpuShares":0,"CpusetCpus":"","CpusetMems":"","DeviceCgroupRules":null,"DeviceRequests":null,"Devices":[],"Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":null,"GroupAdd":null,"IOMaximumBandwidth":0,"IOMaximumIOps":0,"IpcMode":"","Isolation":"","Links":null,"LogConfig":{"Config":{},"Type":""},"MaskedPaths":null,"Memory":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":-1,"NanoCpus":0,"NetworkMode":"default","OomKillDisable":false,"OomScoreAdj":0,"PidMode":"","PidsLimit":0,"PortBindings":{},"Privileged":false,"PublishAllPorts":false,"ReadonlyPaths":null,"ReadonlyRootfs":false,"RestartPolicy":{"MaximumRetryCount":0,"Name":"no"},"SecurityOpt":null,"ShmSize":0,"UTSMode":"","Ulimits":null,"UsernsMode":"","VolumeDriver":"","VolumesFrom":null},"Hostname":"","Image":"hello-world","Labels":{},"NetworkingConfig":{"EndpointsConfig":{}},"OnBuild":null,"OpenStdin":false,"StdinOnce":false,"Tty":false,"User":"","Volumes":{"":{}},"WorkingDir":""}

Formatting that request, it looks like it's sending "" as volume name;

"Volumes": {
 "": {}
},

Full request below;

Details

{
 "AttachStderr": true,
 "AttachStdin": false,
 "AttachStdout": true,
 "Cmd": null,
 "Domainname": "",
 "Entrypoint": null,
 "Env": null,
 "HostConfig": {
 "AutoRemove": true,
 "Binds": null,
 "BlkioDeviceReadBps": [],
 "BlkioDeviceReadIOps": [],
 "BlkioDeviceWriteBps": [],
 "BlkioDeviceWriteIOps": [],
 "BlkioWeight": 0,
 "BlkioWeightDevice": [],
 "CapAdd": null,
 "CapDrop": null,
 "Cgroup": "",
 "CgroupParent": "",
 "CgroupnsMode": "",
 "ConsoleSize": [60, 234],
 "ContainerIDFile": "",
 "CpuCount": 0,
 "CpuPercent": 0,
 "CpuPeriod": 0,
 "CpuQuota": 0,
 "CpuRealtimePeriod": 0,
 "CpuRealtimeRuntime": 0,
 "CpuShares": 0,
 "CpusetCpus": "",
 "CpusetMems": "",
 "DeviceCgroupRules": null,
 "DeviceRequests": null,
 "Devices": [],
 "Dns": [],
 "DnsOptions": [],
 "DnsSearch": [],
 "ExtraHosts": null,
 "GroupAdd": null,
 "IOMaximumBandwidth": 0,
 "IOMaximumIOps": 0,
 "IpcMode": "",
 "Isolation": "",
 "Links": null,
 "LogConfig": {
 "Config": {},
 "Type": ""
 },
 "MaskedPaths": null,
 "Memory": 0,
 "MemoryReservation": 0,
 "MemorySwap": 0,
 "MemorySwappiness": -1,
 "NanoCpus": 0,
 "NetworkMode": "default",
 "OomKillDisable": false,
 "OomScoreAdj": 0,
 "PidMode": "",
 "PidsLimit": 0,
 "PortBindings": {},
 "Privileged": false,
 "PublishAllPorts": false,
 "ReadonlyPaths": null,
 "ReadonlyRootfs": false,
 "RestartPolicy": {
 "MaximumRetryCount": 0,
 "Name": "no"
 },
 "SecurityOpt": null,
 "ShmSize": 0,
 "UTSMode": "",
 "Ulimits": null,
 "UsernsMode": "",
 "VolumeDriver": "",
 "VolumesFrom": null
 },
 "Hostname": "",
 "Image": "hello-world",
 "Labels": {},
 "NetworkingConfig": {
 "EndpointsConfig": {}
 },
 "OnBuild": null,
 "OpenStdin": false,
 "StdinOnce": false,
 "Tty": false,
 "User": "",
 "Volumes": {
 "": {}
 },
 "WorkingDir": ""
}

If I remove the --rm from the docker run;

docker run -v "" hello-world

docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory: unknown.

docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
74b26d0d723a hello-world "/hello" 3 minutes ago Created awesome_mcnulty

When inspecting that container's config;

docker container inspect awesome_mcnulty

There's also the "Volumes": {"": {}}, (as expected), but in addition, the Mounts contains an entry to mount an anonymous volume;

"Mounts": [
 {
 "Type": "volume",
 "Name": "a8a603cb41b14aa42ef9969998fe709db843f2fffec88fdf1d9b162cf9767287",
 "Source": "/var/lib/docker/volumes/a8a603cb41b14aa42ef9969998fe709db843f2fffec88fdf1d9b162cf9767287/_data",
 "Destination": ".",
 "Driver": "local",
 "Mode": "",
 "RW": true,
 "Propagation": ""
 }
],

Full inspect output:

Details

[
 {
 "Id": "74b26d0d723a120b9d5afcdbfdbbfa2c657c674d98ca24e96ff10aef53e9cc3b",
 "Created": "2023-02-27T16:21:00.860555084Z",
 "Path": "/hello",
 "Args": [],
 "State": {
 "Status": "created",
 "Running": false,
 "Paused": false,
 "Restarting": false,
 "OOMKilled": false,
 "Dead": false,
 "Pid": 0,
 "ExitCode": 127,
 "Error": "failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory: unknown",
 "StartedAt": "0001-01-01T00:00:00Z",
 "FinishedAt": "0001-01-01T00:00:00Z"
 },
 "Image": "sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412",
 "ResolvConfPath": "/var/lib/docker/containers/74b26d0d723a120b9d5afcdbfdbbfa2c657c674d98ca24e96ff10aef53e9cc3b/resolv.conf",
 "HostnamePath": "/var/lib/docker/containers/74b26d0d723a120b9d5afcdbfdbbfa2c657c674d98ca24e96ff10aef53e9cc3b/hostname",
 "HostsPath": "/var/lib/docker/containers/74b26d0d723a120b9d5afcdbfdbbfa2c657c674d98ca24e96ff10aef53e9cc3b/hosts",
 "LogPath": "/var/lib/docker/containers/74b26d0d723a120b9d5afcdbfdbbfa2c657c674d98ca24e96ff10aef53e9cc3b/74b26d0d723a120b9d5afcdbfdbbfa2c657c674d98ca24e96ff10aef53e9cc3b-json.log",
 "Name": "/awesome_mcnulty",
 "RestartCount": 0,
 "Driver": "overlay2",
 "Platform": "linux",
 "MountLabel": "",
 "ProcessLabel": "",
 "AppArmorProfile": "",
 "ExecIDs": null,
 "HostConfig": {
 "Binds": null,
 "ContainerIDFile": "",
 "LogConfig": {
 "Type": "json-file",
 "Config": {}
 },
 "NetworkMode": "default",
 "PortBindings": {},
 "RestartPolicy": {
 "Name": "no",
 "MaximumRetryCount": 0
 },
 "AutoRemove": false,
 "VolumeDriver": "",
 "VolumesFrom": null,
 "ConsoleSize": [
 60,
 234
 ],
 "CapAdd": null,
 "CapDrop": null,
 "CgroupnsMode": "private",
 "Dns": [],
 "DnsOptions": [],
 "DnsSearch": [],
 "ExtraHosts": null,
 "GroupAdd": null,
 "IpcMode": "private",
 "Cgroup": "",
 "Links": null,
 "OomScoreAdj": 0,
 "PidMode": "",
 "Privileged": false,
 "PublishAllPorts": false,
 "ReadonlyRootfs": false,
 "SecurityOpt": null,
 "UTSMode": "",
 "UsernsMode": "",
 "ShmSize": 67108864,
 "Runtime": "runc",
 "Isolation": "",
 "CpuShares": 0,
 "Memory": 0,
 "NanoCpus": 0,
 "CgroupParent": "",
 "BlkioWeight": 0,
 "BlkioWeightDevice": [],
 "BlkioDeviceReadBps": [],
 "BlkioDeviceWriteBps": [],
 "BlkioDeviceReadIOps": [],
 "BlkioDeviceWriteIOps": [],
 "CpuPeriod": 0,
 "CpuQuota": 0,
 "CpuRealtimePeriod": 0,
 "CpuRealtimeRuntime": 0,
 "CpusetCpus": "",
 "CpusetMems": "",
 "Devices": [],
 "DeviceCgroupRules": null,
 "DeviceRequests": null,
 "MemoryReservation": 0,
 "MemorySwap": 0,
 "MemorySwappiness": null,
 "OomKillDisable": null,
 "PidsLimit": null,
 "Ulimits": null,
 "CpuCount": 0,
 "CpuPercent": 0,
 "IOMaximumIOps": 0,
 "IOMaximumBandwidth": 0,
 "MaskedPaths": [
 "/proc/asound",
 "/proc/acpi",
 "/proc/kcore",
 "/proc/keys",
 "/proc/latency_stats",
 "/proc/timer_list",
 "/proc/timer_stats",
 "/proc/sched_debug",
 "/proc/scsi",
 "/sys/firmware"
 ],
 "ReadonlyPaths": [
 "/proc/bus",
 "/proc/fs",
 "/proc/irq",
 "/proc/sys",
 "/proc/sysrq-trigger"
 ]
 },
 "GraphDriver": {
 "Data": {
 "LowerDir": "/var/lib/docker/overlay2/b2e919471d78b621b3b86fd9c6c08bc874ba6db48b820a8e625305919f5afa46-init/diff:/var/lib/docker/overlay2/f5fbe8ccb18d1f750d09eb21d904ecff10dcf44656e0e1655f2fb1f09b90506e/diff",
 "MergedDir": "/var/lib/docker/overlay2/b2e919471d78b621b3b86fd9c6c08bc874ba6db48b820a8e625305919f5afa46/merged",
 "UpperDir": "/var/lib/docker/overlay2/b2e919471d78b621b3b86fd9c6c08bc874ba6db48b820a8e625305919f5afa46/diff",
 "WorkDir": "/var/lib/docker/overlay2/b2e919471d78b621b3b86fd9c6c08bc874ba6db48b820a8e625305919f5afa46/work"
 },
 "Name": "overlay2"
 },
 "Mounts": [
 {
 "Type": "volume",
 "Name": "a8a603cb41b14aa42ef9969998fe709db843f2fffec88fdf1d9b162cf9767287",
 "Source": "/var/lib/docker/volumes/a8a603cb41b14aa42ef9969998fe709db843f2fffec88fdf1d9b162cf9767287/_data",
 "Destination": ".",
 "Driver": "local",
 "Mode": "",
 "RW": true,
 "Propagation": ""
 }
 ],
 "Config": {
 "Hostname": "74b26d0d723a",
 "Domainname": "",
 "User": "",
 "AttachStdin": false,
 "AttachStdout": true,
 "AttachStderr": true,
 "Tty": false,
 "OpenStdin": false,
 "StdinOnce": false,
 "Env": [
 "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 ],
 "Cmd": [
 "/hello"
 ],
 "Image": "hello-world",
 "Volumes": {
 "": {}
 },
 "WorkingDir": "",
 "Entrypoint": null,
 "OnBuild": null,
 "Labels": {}
 },
 "NetworkSettings": {
 "Bridge": "",
 "SandboxID": "5258a296cff5bfc90a8fe433665473bcae77698417b2cd9aa4eed6b6598452b3",
 "HairpinMode": false,
 "LinkLocalIPv6Address": "",
 "LinkLocalIPv6PrefixLen": 0,
 "Ports": {},
 "SandboxKey": "/var/run/docker/netns/5258a296cff5",
 "SecondaryIPAddresses": null,
 "SecondaryIPv6Addresses": null,
 "EndpointID": "",
 "Gateway": "",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "IPAddress": "",
 "IPPrefixLen": 0,
 "IPv6Gateway": "",
 "MacAddress": "",
 "Networks": {
 "bridge": {
 "IPAMConfig": null,
 "Links": null,
 "Aliases": null,
 "NetworkID": "b6a34a8f06831306aef0624edacb073427d46fa866a652ef61cc998a5fe3d038",
 "EndpointID": "",
 "Gateway": "",
 "IPAddress": "",
 "IPPrefixLen": 0,
 "IPv6Gateway": "",
 "GlobalIPv6Address": "",
 "GlobalIPv6PrefixLen": 0,
 "MacAddress": "",
 "DriverOpts": null
 }
 }
 }
 }
]

And looking at the list of volumes, there's indeed a volume created;

docker volume ls
DRIVER VOLUME NAME
local a8a603cb41b14aa42ef9969998fe709db843f2fffec88fdf1d9b162cf9767287

So what looks to be happening;

• The CLI sends a -v "" (volume with an empty name), but destination set for inside the container
• The daemon creates an anonymous volume (as no name was specified for the volume to be created)
• The daemon sets . (current directory) as mount-point for the volume
• This information gets sent to the OCI runtime (runc), which mounts the volume to . (/), effectively "masking" the container's whole rootfs (mounting an empty directory over /), which then fails (as paths such as /proc/sys/net/ipv4/ip_unprivileged_port_start are masked, and thus cannot be found inside the container's rootfs).

It looks like there's some potential other corner-cases as well; for example: "":/ is treated as a literal :/ as mount-point (which gets normalised to :);

docker run -it -v "":/ hello-world

"Volumes": {
 ":/": {}
},

"Mounts": [
 {
 "Type": "volume",
 "Name": "a164673e992aa46c8099b2bd34dcde4ed037a5b3c7290b6cb056bafbd8b99213",
 "Source": "/var/lib/docker/volumes/a164673e992aa46c8099b2bd34dcde4ed037a5b3c7290b6cb056bafbd8b99213/_data",
 "Destination": ":",
 "Driver": "local",
 "Mode": "",
 "RW": true,
 "Propagation": ""
 }
],

But using a proper name for the volume, and / as target has special handling;

docker run -it -v myvolume:/ hello-world
docker: Error response from daemon: invalid volume specification: 'myvolume:/': invalid mount config for type "volume": invalid specification: destination can't be '/'.
See 'docker run --help'.

I seem to recall I had a PR to better normalize the destination (including making sure that . also produces that error), but it wasn't accepted at the time (perhaps should be revisited).

[thaJeztah]

related;

• Validation of volumes (both during build, and run) is incomplete moby/moby#35879
• Check that the VOLUME instruction arguments are not '.' or '/' moby/moby#35981