[BUG] docker compose up --watch with sync+restart does not show logs after restart

[scooper91]

Description

When running docker compose up --watch, the logs for the service appear in the output.

When specifying sync+restart as the action on a file change, when a file is changed, the container restarts, but there are no more logs outputted. When running docker compose logs, you can see the app was restarted correctly and is still logging.

This behaviour does not happen when using rebuild as the action (although the logs aren't picked up straight away so the first logs are potentially missed).

Expected behaviour is that when using sync+restart, when a file is changed, the logs from the container continue to be shown in the output of docker compose up --watch.

Steps To Reproduce

Dockerfile:

FROM node:slim
COPY src src

Contents of src can be anything.

Docker Compose:

services:
 app:
 build: .
 command: >-
 bash -ceuo pipefail "
 echo restart
 i=0
 while true; do echo boop $$i && ((i++)) && sleep 1; done
 "
 stop_signal: SIGKILL
 develop:
 watch:
 - action: sync+restart
 path: ./src
 target: /src

Running docker compose up --watch, then saving a file in src outputs:

➜ docker compose up --watch
[+] Running 1/2
 ✔ Network docker-compose-watch_default Created 0.1s 
 ⠇ Container docker-compose-watch-app-1 Created 0.8s 
 ⦿ Watch enabled
Attaching to app-1
app-1 | restart
app-1 | boop 0
app-1 | boop 1
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
app-1 | boop 6
 ⦿ Syncing "app" after changes were detected
[+] Restarting 1/1
 ✔ Container docker-compose-watch-app-1 Started 0.6s 
app-1 exited with code 0

When running docker compose logs, it shows all the logs correctly.

➜ docker compose logs 
app-1 | restart 
app-1 | boop 0 
app-1 | boop 1 
app-1 | boop 2 
app-1 | boop 3 
app-1 | boop 4 
app-1 | boop 5 
app-1 | boop 6 
app-1 | restart 
app-1 | boop 0 
app-1 | boop 1 
app-1 | boop 2 
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5

If you change sync+restart to rebuild in the docker-compose file, and run docker compose up --watch, and save a file, it now prints (most of) the logs correctly:

➜ docker compose up --watch
[+] Running 1/2
 ✔ Network docker-compose-watch_default Created 0.1s 
 ⠧ Container docker-compose-watch-app-1 Created 0.7s 
 ⦿ Watch enabled
Attaching to app-1
app-1 | restart
app-1 | boop 0
app-1 | boop 1
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
 ⦿ Rebuilding service "app" after changes were detected...
app-1 | boop 6
 ⦿ service "app" successfully built
app-1 exited with code 0
app-1 has been recreated
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
app-1 | boop 6

Note that in the output, it misses the first few logs.

Doing a docker compose logs only shows the logs for the latest container.

Compose Version

➜ docker compose version
Docker Compose version v2.26.1


```sh
➜ docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019

### Docker Environment

```Text
➜ docker info 
Client: Docker Engine - Community 
 Version: 26.1.0
 Context: default
 Debug Mode: false
 Plugins:
 buildx: Docker Buildx (Docker Inc.)
 Version: v0.14.0
 Path: /usr/libexec/docker/cli-plugins/docker-buildx
 compose: Docker Compose (Docker Inc.)
 Version: v2.26.1
 Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
 Images: 125
 Server Version: 26.1.0
 Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Using metacopy: false
 Native Overlay Diff: true
 userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
 apparmor
 seccomp
 Profile: builtin
 cgroupns
 Kernel Version: 6.5.0-27-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.31GiB
 Name: --
 ID: QBED:UL4T:4VZM:D6KX:NMXE:WYFE:6DDN:PK4Y:CEGA:C5IZ:WBPG:JZES
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
 127.0.0.0/8
 Live Restore Enabled: false

Anything else?

Feature introduction and demo

[jbne]

Similar behavior happens when sending a docker compose up --watch job to the background and watching the output with docker compose logs --follow in another command window. The service logs show up until the service restart occurs, after which the logs command exits with code 0. One might expect docker compose logs --follow to show the sync/restart messages along with the rest of the output from the restarted (or terminated and started again) service.

[dkaste]

I think this happens when the number of running services drops to zero, even if temporarily for a restart. For example, if you start two services but one exits due to an error, this issue will occur when the other one is restarted.

I was able to work around this issue for now by keeping a dummy service alive at all times:

dummy-watch-workaround:
 image: alpine:latest
 init: true
 command: ["sh", "-c", "while true; do sleep 2; done"]

[Hellzed]

@dkaste I confirm. Let's say I have a db and a web service, the web service being the one I want to watch in sync+restart mode.

Just running docker compose up --watch web ends up with the terminal detaching from the service with web-1 exited with code 0.

But running docker compose up --watch db web behaves correctly.

[torarnv]

I'm still seeing this with 2.30.3

❯ docker compose up --watch
[+] Running 1/0
 ✔ Container caddy-caddy-1 Running 0.0s
 ⦿ Watch enabled
Attaching to caddy-1
 ⦿ Syncing and restarting service "caddy" after 1 changes were detected
caddy-1 | 2024/11/23 00:42:30.609	INFO	shutting down apps, then terminating	{"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:30.609	WARN	exiting; byeee!! 👋	{"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:30.609	INFO	http	servers shutting down with eternal grace period
caddy-1 | 2024/11/23 00:42:30.609	INFO	admin	stopped previous server	{"address": "localhost:2019"}
caddy-1 | 2024/11/23 00:42:30.609	INFO	shutdown complete	{"signal": "SIGTERM", "exit_code": 0}
 ⦿ service "caddy" restarted
caddy-1 exited with code 0
 ⦿ Syncing and restarting service "caddy" after 1 changes were detected
caddy-1 | 2024/11/23 00:42:39.434	INFO	shutting down apps, then terminating	{"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:39.434	WARN	exiting; byeee!! 👋	{"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:39.434	INFO	http	servers shutting down with eternal grace period
caddy-1 | 2024/11/23 00:42:39.434	INFO	admin	stopped previous server	{"address": "localhost:2019"}
caddy-1 | 2024/11/23 00:42:39.434	INFO	shutdown complete	{"signal": "SIGTERM", "exit_code": 0}
caddy-1 exited with code 0
 ⦿ Watch disabled

❯

After one or two changes to the Caddyfile, the service is restarted, and keeps running, but the watch disconnects.

services:
 caddy:
 image: torarnv/caddy
 build: .
 restart: unless-stopped
 ports:
 - "8080:80"

 develop:
 watch:
 - action: sync+restart
 path: ./Caddyfile
 target: /etc/caddy/Caddyfile

[torarnv]

Can someone please re-open this?

[Tarasovych]

Same issue, occurrs from time to time after container recreation