PostgreSQL CDC

A simple PostgreSQL Change Data Capture (CDC) client in Rust that streams and parses database changes (INSERT, UPDATE, DELETE, TRUNCATE, RELATION) to JSON output in real-time using logical replication.

Setup

1. Configure PostgreSQL

Edit postgresql.conf:

wal_level = logical
max_replication_slots = 4
max_wal_senders = 4

Restart PostgreSQL.

2. Grant Replication Privileges

ALTERUSER your_user WITH REPLICATION;

3. Create Database

CREATEDATABASEcdc_test;

4. Create Publication

psql cdc_test

CREATE PUBLICATION cdc_publication FOR ALL TABLES;

5. Create Replication Slot

psql "dbname=cdc_test replication=database"

CREATE_REPLICATION_SLOT cdc_slot LOGICAL pgoutput;

Default Environment Variables

PG_HOST=localhost
PG_PORT=5432
PG_USER=postgres
PG_PASSWORD=postgres
PG_DATABASE=cdc_test
PG_PUBLICATION=cdc_publication
PG_REPLICATION_SLOT=cdc_slot

Run

cargo run

References

• https://www.postgresql.org/docs/current/sql-createpublication.html
• https://www.postgresql.org/docs/current/warm-standby.html#STREAMING-REPLICATION-SLOTS
• https://www.postgresql.org/docs/current/protocol-replication.html
• https://www.postgresql.org/docs/current/protocol-logicalrep-message-formats.html