[BUG] Container named volume mount source not getting updated

[g-kartik]

Description

I want postgres database to use separate data volumes for each git branch. I am passing the CURRENT_GIT_BRANCH value through .env file.

Here is the docker compose file

services:
 cvat_db:
 image: postgres:15-alpine
 environment:
 POSTGRES_USER: root
 POSTGRES_DB: cvat
 POSTGRES_HOST_AUTH_METHOD: trust
 volumes:
 - cvat_db:/var/lib/postgresql/data

volumes:
 cvat_db:
 name: cvat_db_{CURRENT_GIT_BRANCH}

When executing docker compose up for the first time with CURRENT_GIT_BRANCH as develop, the container gets correctly mounted with the volume cvat_db_develop.

With a new value for the CURRENT_GIT_BRANCH, say feature, the volume cvat_db_feature gets created successfully, however the container volume mount source still refers to the old cvat_db_develop volume.

The expected behavior is that by changing the name of the named volume:

• A new volume should be created with the given name (happens correctly)
• The source mount path of all the containers referencing that volume should refer to the new volume. (does not happen)

Compose Version

Docker Compose version v2.24.5-desktop.1

Docker Environment

Client:
 Version: 24.0.7
 Context: default
 Debug Mode: false
 Plugins:
 buildx: Docker Buildx (Docker Inc.)
 Version: v0.12.1-desktop.4
 Path: /usr/lib/docker/cli-plugins/docker-buildx
 compose: Docker Compose (Docker Inc.)
 Version: v2.24.5-desktop.1
 Path: /usr/lib/docker/cli-plugins/docker-compose
 debug: Get a shell into any image or container. (Docker Inc.)
 Version: 0.0.24
 Path: /usr/lib/docker/cli-plugins/docker-debug
 dev: Docker Dev Environments (Docker Inc.)
 Version: v0.1.0
 Path: /usr/lib/docker/cli-plugins/docker-dev
 extension: Manages Docker extensions (Docker Inc.)
 Version: v0.2.21
 Path: /usr/lib/docker/cli-plugins/docker-extension
 feedback: Provide feedback, right in your terminal! (Docker Inc.)
 Version: v1.0.4
 Path: /usr/lib/docker/cli-plugins/docker-feedback
 init: Creates Docker-related starter files for your project (Docker Inc.)
 Version: v1.0.0
 Path: /usr/lib/docker/cli-plugins/docker-init
 sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
 Version: 0.6.0
 Path: /usr/lib/docker/cli-plugins/docker-sbom
 scout: Docker Scout (Docker Inc.)
 Version: v1.4.1
 Path: /usr/lib/docker/cli-plugins/docker-scout

Server:
 Containers: 8
 Running: 8
 Paused: 0
 Stopped: 0
 Images: 22
 Server Version: 24.0.7
 Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Using metacopy: true
 Native Overlay Diff: false
 userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 64b8a811b07ba6288238eefc14d898ee0b5b99ba.m
 runc version: 
 init version: de40ad0
 Security Options:
 seccomp
 Profile: builtin
 cgroupns
 Kernel Version: 6.6.5-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 19.45GiB
 Name: archlinux
 ID: c755af18-ccd5-4262-9be4-a17d19c101b2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: gkarthikraja
 Experimental: false
 Insecure Registries:
 127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[ndeloof]

Only services indeed get recreated after configuration change has been detected. We could do the same for volumes (while bit more complex to manage regarding lifecycle) but this would mean user data would be deleted, which could be an issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[g-kartik]

@ndeloof
Can we keep the old volumes without deleting and mount the new one?

[ndeloof]

@g-kartik nope, as volume name is unique we have no option to keep the existing one

[g-kartik]

I did not understand. The issue is regarding when volume names changes in the compose file, the docker compose up command should unmount (but not delete) the old volume and mount the new volume. If we again change back to old volume then it should get mounted again and reused. There is no issue of uniqueness here.

[ndeloof]

right, my bad, have been reading too fast.
The issue here is that compose compares actual container state with desired state only based on service.foo yaml section, where volume is referenced as cvat_db in the reported example. The fact actual volume name is updated (or any other configuration attribute) isn't detected. A comparable issue is #11900

IMHO we miss a global mechanism to detect volume/config/secrets/networks do not run with desired state, and recreate services accordingly

[ndeloof]

this has been fixed by #12363