Docker Compose Backup and Server Migration Guide

English | فارسی

A general-purpose, safe process for moving Docker Compose projects from an old server to a new one.

The guide focuses on:

• PostgreSQL
• Docker Volumes
• Media
• Compression with gzip and zstd
• Backup validation
• Transfer with rsync
• Restore
• Preventing data loss
• Final testing and cutover

---

1. Defining Project Variables

So that the commands don't depend on any one specific project, we define variables first:

PROJECT_NAME="myproject"
DB_NAME="mydatabase"
DB_USER="postgres"

MEDIA_VOLUME="${PROJECT_NAME}_media_volume"
POSTGRES_VOLUME="${PROJECT_NAME}_postgres_data"

BACKUP_DIR="/backup/${PROJECT_NAME}"

Every command from here on uses these variables.

A note on -T in docker compose exec

Every command in this guide uses -T:

docker compose -f docker-compose.prod.yml exec -T db ...

Without -T, Docker allocates a TTY and corrupts the binary output of pg_dump. The result is a file that appears to have been created but cannot be restored.

When transferring binary data, -T is not optional.

---

2. Initial Survey of the Old Server

Before starting the backup, we need to know exactly which services and volumes we have.

Checking services

docker compose -f docker-compose.prod.yml ps

Checking volumes

docker volume ls

Checking Docker disk usage

docker system df -v

Important:

Before migrating, you must know exactly which volumes hold the database, the media, and any other persistent data.

---

3. Backing Up the PostgreSQL Database

For PostgreSQL, the primary backup method is pg_dump.

Using the custom format with -Fc is recommended:

mkdir -p "$BACKUP_DIR"

docker compose -f docker-compose.prod.yml exec -T db \
  pg_dump -U "$DB_USER" -d "$DB_NAME" -Fc \
  > "$BACKUP_DIR/postgres.dump"

But since we intend to transfer the backup file, it is better to compress it.

---

4. Compressing PostgreSQL with gzip

First approach:

docker compose -f docker-compose.prod.yml exec -T db \
  pg_dump -U "$DB_USER" -d "$DB_NAME" -Fc \
  | gzip -9 \
  > "$BACKUP_DIR/postgres.dump.gz"

The structure:

PostgreSQL
    ↓
pg_dump -Fc
    ↓
gzip -9
    ↓
postgres.dump.gz

Validating gzip

Always test the compressed file itself:

gzip -t "$BACKUP_DIR/postgres.dump.gz"

If the command finishes without error, the gzip file is intact.

Checking the PostgreSQL dump structure

gzip -dc "$BACKUP_DIR/postgres.dump.gz" \
  | pg_restore --list > /dev/null

Beyond confirming the gzip file is intact, this test verifies that the dump's contents are readable by pg_restore.

---

5. Compressing PostgreSQL with zstd

For large backups, zstd is an excellent choice.

docker compose -f docker-compose.prod.yml exec -T db \
  pg_dump -U "$DB_USER" -d "$DB_NAME" -Fc \
  | zstd -T0 -10 -f \
  -o "$BACKUP_DIR/postgres.dump.zst"

What the options mean

-T0

Use all available CPU threads.

-10

Compression level.

-f

Allow overwriting an existing output file. Without it, re-running the backup stops with an error.

For more speed you could use, for example:

zstd -T0 -5

And for better compression:

zstd -T0 -10

is a good choice.

---

6. Validating a PostgreSQL Dump Compressed with zstd

First we test the zstd file itself:

zstd -t "$BACKUP_DIR/postgres.dump.zst"

Then we check the dump structure:

zstd -dc "$BACKUP_DIR/postgres.dump.zst" \
  | pg_restore --list > /dev/null

So for PostgreSQL with zstd we have two tests:

postgres.dump.zst
       │
       ├── zstd -t
       │
       └── pg_restore --list

---

7. gzip or zstd?

Feature	gzip	zstd
Compression	Good	Very good
Compression speed	Moderate	Usually faster
Decompression speed	Good	Very good
Multi-thread	Limited	Yes
Suitable for large backups	Good	Excellent
Validation	gzip -t	zstd -t

For large backups, my recommendation:

pg_dump
   ↓
zstd -T0 -10
   ↓
postgres.dump.zst

---

8. Backing Up the Media Volume

Media typically includes:

• Images
• PDFs
• User files
• Attachments
• Uploaded files

To back it up, we can mount the volume into a temporary container:

docker run --rm \
  -v "$MEDIA_VOLUME":/data:ro \
  alpine \
  tar -cf - -C /data . \
  | zstd -T0 -10 -f \
  -o "$BACKUP_DIR/media.tar.zst"

Note: zstd runs here on the host, not inside the container. That is why there is no need to mount $BACKUP_DIR into the container, and why zstd must be installed on the host.

The structure:

Docker Volume
     ↓
    tar
     ↓
   zstd
     ↓
media.tar.zst

Using :ro matters, because the backup container must not be able to modify the original data.

---

9. Validating the Media Backup

First we check the zstd file:

zstd -t "$BACKUP_DIR/media.tar.zst"

Then we check the archive itself:

zstd -dc "$BACKUP_DIR/media.tar.zst" \
  | tar -tf - > /dev/null

So:

media.tar.zst
     │
     ├── zstd -t
     │
     └── tar -tf

Both must run without error.

---

10. Optional Backup of the Database Volume

If needed, we can also back up the PostgreSQL volume:

docker run --rm \
  -v "$POSTGRES_VOLUME":/data:ro \
  alpine \
  tar -cf - -C /data . \
  | zstd -T0 -10 -f \
  -o "$BACKUP_DIR/postgres-volume.tar.zst"

And then:

zstd -t "$BACKUP_DIR/postgres-volume.tar.zst"

But keep in mind:

A raw volume backup is not a replacement for pg_dump.

For a logical PostgreSQL backup, the primary method is still:

pg_dump

A volume backup is more useful as a supplementary backup or for disaster recovery.

---

11. Creating SHA256SUMS

Once the backup files have been confirmed structurally sound, we hash them:

cd "$BACKUP_DIR"

sha256sum *.zst > SHA256SUMS

If you used gzip instead:

sha256sum *.gz > SHA256SUMS

Writing *.gz *.zst together is not correct; if either pattern matches no files, sha256sum errors out.

Then:

sha256sum -c SHA256SUMS

The output should look like this:

postgres.dump.gz: OK
postgres.dump.zst: OK
media.tar.zst: OK
postgres-volume.tar.zst: OK

Naturally, only the files that actually exist will appear in the output.

---

12. The Three Validation Layers

A trustworthy backup must have at least three layers of validation.

Layer one: compression test

For gzip:

gzip -t postgres.dump.gz

For zstd:

zstd -t postgres.dump.zst

---

Layer two: archive / dump test

For PostgreSQL:

zstd -dc postgres.dump.zst \
  | pg_restore --list > /dev/null

For media:

zstd -dc media.tar.zst \
  | tar -tf - > /dev/null

---

Layer three: SHA256

sha256sum -c SHA256SUMS

So:

Compression OK
       +
Archive/Dump OK
       +
SHA256 OK
       ↓
Backup Ready

---

13. Transferring the Backup with rsync

After validation, we transfer the backup to the new server:

rsync -avhP \
  "$BACKUP_DIR/" \
  user@NEW_SERVER:"$BACKUP_DIR/"

The -P option means:

• Progress is displayed.
• Interrupted transfers resume more easily.

You may see an error such as:

failed to set times ... Operation not permitted

On its own, this error does not mean the files are corrupt.

After the transfer, always verify the hashes.

---

14. Validation on the New Server

On the new server, we first have to redefine the same variables from section 1; the shell variables from the old server do not exist here:

PROJECT_NAME="myproject"
DB_NAME="mydatabase"
DB_USER="postgres"

MEDIA_VOLUME="${PROJECT_NAME}_media_volume"
POSTGRES_VOLUME="${PROJECT_NAME}_postgres_data"

BACKUP_DIR="/backup/${PROJECT_NAME}"

cd "$BACKUP_DIR"

Then:

sha256sum -c SHA256SUMS

Then the compressed files:

gzip -t postgres.dump.gz

or:

zstd -t postgres.dump.zst

and:

zstd -t media.tar.zst

Then the dump structure:

zstd -dc postgres.dump.zst \
  | pg_restore --list > /dev/null

And the media structure:

zstd -dc media.tar.zst \
  | tar -tf - > /dev/null

This step proves the backup was not corrupted or altered in transit.

---

15. Preparing Docker Compose on the New Server

First transfer the project and check the configuration:

docker compose -f docker-compose.prod.yml config

Then bring up the database:

docker compose -f docker-compose.prod.yml up -d db

This also creates the required volume.

---

16. Restoring PostgreSQL from zstd

If our backup is:

postgres.dump.zst

then:

zstd -dc "$BACKUP_DIR/postgres.dump.zst" \
  | docker compose -f docker-compose.prod.yml exec -T db \
  pg_restore \
    -U "$DB_USER" \
    -d "$DB_NAME" \
    --clean \
    --if-exists

An important note about --clean

Before using:

--clean

we must be certain the target database is genuinely the one meant to be replaced.

---

17. Restoring PostgreSQL from gzip

If we used:

postgres.dump.gz

then:

gzip -dc "$BACKUP_DIR/postgres.dump.gz" \
  | docker compose -f docker-compose.prod.yml exec -T db \
  pg_restore \
    -U "$DB_USER" \
    -d "$DB_NAME" \
    --clean \
    --if-exists

---

18. Restoring Media

The default alpine image does not include zstd:

docker run --rm alpine sh -c 'command -v zstd'

The output is empty. So we install zstd first:

docker run --rm \
  -v "$MEDIA_VOLUME":/data \
  -v "$BACKUP_DIR":/backup:ro \
  alpine \
  sh -c 'apk add --no-cache zstd && \
         zstd -dc /backup/media.tar.zst | tar -xf - -C /data'

Note: apk add requires internet access on the new server.

Then we check the volume size:

docker run --rm \
  -v "$MEDIA_VOLUME":/data:ro \
  alpine \
  du -sh /data

The size should roughly match the old server.

---

19. Bringing Up the Services

After the restore:

docker compose -f docker-compose.prod.yml up -d

Then:

docker compose -f docker-compose.prod.yml ps

All of the main services should be running.

---

20. Checking Redis

docker compose -f docker-compose.prod.yml exec redis redis-cli ping

Expected output:

PONG

---

21. Checking Celery

docker compose -f docker-compose.prod.yml logs --tail=100 celery_worker

and:

docker compose -f docker-compose.prod.yml logs --tail=100 celery_beat

There should be no connection errors to Redis or PostgreSQL.

---

22. Checking the Application

Logs:

docker compose -f docker-compose.prod.yml logs --tail=100 app

If you have services such as myproject_app:

docker compose -f docker-compose.prod.yml logs --tail=100 myproject_app

We also need to verify the application is listening on the right interface.

In Docker it should usually be:

0.0.0.0:8000

not:

127.0.0.1:8000

For example:

docker compose -f docker-compose.prod.yml exec myproject_app \
  sh -c 'env | sort | grep -Ei "gunicorn|bind|worker|config"'

---

23. Checking Nginx

Before reloading:

nginx -t

If it succeeds:

systemctl reload nginx

Also check the logs:

docker compose -f docker-compose.prod.yml logs --tail=100 nginx

Or, if Nginx is installed on the host:

tail -f /var/log/nginx/error.log

---

24. Checking DNS and SSL

Before running Certbot, make sure DNS is correct for all domains.

For example:

dig +short example.com
dig +short www.example.com

And for subdomains as well:

dig +short panel.example.com
dig +short api.example.com

If a domain does not exist, Certbot may return:

NXDOMAIN

So the correct order is:

DNS
 ↓
Server
 ↓
Nginx
 ↓
HTTP
 ↓
Certbot / SSL

---

25. End-to-End Testing

Containers being in a running state is not enough.

We have to test the real application.

Authentication

• Login
• Logout
• JWT / Session

Database

• Older records
• Users
• Core data
• Record counts

Media

• Older images
• PDFs
• Uploaded files
• Download
• New upload

Background Jobs

• Celery Worker
• Celery Beat
• Redis

Network

• Nginx
• HTTPS
• Domain
• API

---

26. Comparing the Old and New Servers

For extra confidence, compare a few metrics before and after:

Number of users
Number of core records
Number of files
Media size
Database size

For example:

Old Server
-----------
Users:       X
Records:     Y
Media:       Z GB


New Server
-----------
Users:       X
Records:     Y
Media:       Z GB

The goal is that it is not merely:

Container = Running

but rather:

Data = Correct

---

27. Preventing Data Loss

The most important step is taking a correct final backup right before cutover.

The recommended process:

Old Server
    ↓
Stop Writes / Maintenance
    ↓
Final pg_dump
    ↓
Final Media Backup
    ↓
Compression Test
    ↓
Archive/Dump Test
    ↓
SHA256
    ↓
Transfer
    ↓
New Server
    ↓
SHA256 Validation
    ↓
Restore
    ↓
Application Test
    ↓
DNS Cutover

This minimizes the gap between the final backup and the moment traffic moves over.

---

28. Common Mistakes

Just copying the volume

Copying the volume is not a replacement for pg_dump.

For PostgreSQL:

pg_dump
=
the primary logical backup

and:

postgres volume backup
=
supplementary backup / disaster recovery

---

Trusting the file size

The fact that a backup file exists and has a reasonable size does not mean it is intact.

For example:

ls -lh postgres.dump.zst

is not enough.

You must run:

zstd -t postgres.dump.zst

and:

pg_restore --list

and:

sha256sum -c SHA256SUMS

---

Deleting the old server too quickly

After a successful migration, do not delete the old server immediately.

It is better to keep it around for a while in case you need to roll back.

---

Forgetting .env

Items such as:

DATABASE_URL
SECRET_KEY
REDIS_URL
ALLOWED_HOSTS
CORS
JWT configuration
Storage configuration
API keys

all need to be checked.

---

29. Final Checklist

• All Docker Compose projects have been identified.
• All important volumes have been identified.
• A PostgreSQL pg_dump has been taken.
• The dump has been compressed with gzip or zstd.
• gzip -t or zstd -t succeeded.
• pg_restore --list succeeded.
• A media backup has been taken.
• zstd -t media.tar.zst succeeded.
• tar -tf succeeded.
• SHA256SUMS has been created.
• SHA256 is OK on the old server.
• The backup has been transferred with rsync.
• SHA256 is OK on the new server.
• The compression test succeeded on the new server.
• PostgreSQL has been restored.
• Media has been restored.
• Redis responds with PONG.
• Celery Worker is healthy.
• Celery Beat is healthy.
• The application is healthy.
• Nginx is healthy.
• SSL is healthy.
• DNS is correct.
• Login has been tested.
• The API has been tested.
• Older media files have been tested.
• Record counts have been compared.
• The old server has been kept for rollback.

---

30. Process Summary

The full process can be summarized in this pipeline:

                    OLD SERVER
                        │
                        ▼
                  ┌───────────┐
                  │  Backup   │
                  └─────┬─────┘
                        │
          ┌─────────────┴─────────────┐
          ▼                           ▼
      PostgreSQL                    Media
          │                           │
      pg_dump -Fc                    tar
          │                           │
     gzip / zstd                    zstd
          │                           │
          └─────────────┬─────────────┘
                        ▼
                Integrity Tests
                        │
             ┌──────────┴──────────┐
             ▼                     ▼
          gzip -t                zstd -t
             │                     │
             └──────────┬──────────┘
                        ▼
                 Archive Tests
                        │
                ┌───────┴───────┐
                ▼               ▼
          pg_restore --list    tar -tf
                │               │
                └───────┬───────┘
                        ▼
                    SHA256
                        │
                        ▼
                      rsync
                        │
                        ▼
                    NEW SERVER
                        │
                        ▼
                 SHA256 Validation
                        │
                        ▼
                     Restore
                        │
                        ▼
                 Docker Compose
                        │
                        ▼
              Application Testing
                        │
                        ▼
                   DNS Cutover
                        │
                        ▼
                    Monitor
                        │
                        ▼
               Rollback Window

The Golden Rule

The existence of a backup file does not mean the backup is intact.

A trustworthy backup must be:

Compression OK
       +
Archive/Dump OK
       +
SHA256 OK
       +
Restore OK
       +
Application Test OK

For that reason, always keep these three commands in mind:

gzip -t backup.dump.gz

zstd -t backup.dump.zst

sha256sum -c SHA256SUMS

And to be sure about the backup's contents:

zstd -dc backup.dump.zst \
  | pg_restore --list > /dev/null

And for media:

zstd -dc media.tar.zst \
  | tar -tf - > /dev/null

This pattern works for any Docker Compose project and is not tied to any particular project name.