Only recreate what's changed

[aanand]

Closes#1345. To try it out:

$ docker-compose up -d --x-smart-recreate

Here's what needs fixing:

• If the image tag (whether arrived at via image or build) has changed, this should trigger a recreate.
• When a service is recreated, dependent services via links, volumes_from and net:container should also be recreated, whether or not their configuration has changed.
• If a container is up-to-date but not running, it should be started.

[aanand]

Edit: this is now implemented.

After giving it some thought, it'd be far easier to just label every container with a hash generated from the service dict that was used to create it, and compare hashes to see if a container needs recreating.

This means we can avoid doing any of the configuration diffing stuff, which is (a) hairy to implement (b) full of corner cases (c) going to be a total pain to debug.

Thoughts?

[dnephin]

I agree, I like the idea of using a hash. It should make things a lot easier.

[aanand]

Pushed an initial implementation using hashes.

[aanand]

Downstream dependent services are now recreated when an upstream dependency is recreated.

[aanand]

This is now ready for review.

[aanand]

One thing worth noting: the --no-build flag has been rendered somewhat useless, as we now have to hit the API (to determine the image ID) before creating containers. Still, we're calling inspect_image() rather than images(), so perhaps it won't be a huge performance regression.

Compose now shows a warning if you pass --no-build but the image doesn't exist.

[aanand]

Just rebasing this now that #1356 is in!

[dnephin]

When we were chatting about #1395 I suggested to support the feature we'd probably want to store the DAG of the services in compose.project.sort_service_dicts() instead of flattening it immediately.

I think that data structure would make this easier as well. Instead of having to lookup the dependencies for each service, we could immediately add recreate plans for all dependencies. When it's available we could consider using it here.

[aanand]

Yeah, I think that'd be a good refactor as part of implementing #1395.

[aanand]

Rebased and removed converge() from Service.

[aanand]

Whoops, the tests are still using converge(). I'll move it into a test helper in a separate PR.

[aanand]

Build is green.

[dnephin]

LGTM

[prologic]

This feature isn't it any published version of docker-compose yet right?

[wader]

@prologic it was introduced in 1.3

[prologic]

As a default (no special cil option) feature? :)

[wader]

Sorry, not default, as a option to up, see docker-compose up --help

[prologic]

dc up --help
--force-recreate Recreate containers even if their configuration and
image haven't changed. Incompatible with --no-recreate.

This one I guess :)

[wader]

Hmm what version is that?

$ docker-compose up --help | grep recreate
--x-smart-recreate Only recreate containers whose configuration or
--no-recreate If containers already exist, don't recreate them.

[prologic]

$ dc --version
docker-compose version: 1.4.0
$ dc up --help | grep recreate
volumes). To prevent Compose from picking up changes, use the `--no-recreate`
If you want to force Compose to stop and recreate all containers, use the
`--force-recreate` flag.
--force-recreate Recreate containers even if their configuration and
image haven't changed. Incompatible with --no-recreate.
--no-recreate If containers already exist, don't recreate them.

[wader]

Your totally right, i'm running 1.3.1 and should probably update :)

From 1.4 changelog https://github.com/docker/compose/blob/master/CHANGELOG.md

The experimental --x-smart-recreate flag which introduced this feature in Compose 1.3.0 has been removed, and a --force-recreate flag has been added for when you want to recreate everything.

[prologic]

👍 Awesome :)