Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Repository files navigation

pgread

TestcodecovGo Report Card

Dump PostgreSQL data without credentials - if you can read the files, you can dump the database.

Blog post:Dumping PostgreSQL Without Credentials: Heap File Parsing for Offensive Security

The Technique

PostgreSQL uses fixed OIDs for system catalogs:

OIDCatalogPath
1262pg_databaseglobal/1262
1259pg_classbase/<db_oid>/1259
1249pg_attributebase/<db_oid>/1249

Leak these 3 files → discover entire schema → dump any table.

Install

go install github.com/Chocapikk/pgread@latest

CLI

# Basic usage
pgread # Auto-detect and dump (JSON)
pgread -table # Output as psql-style table
pgread -sql # Output as SQL statements
pgread -csv # Output as CSV
pgread -sql -db mydb > backup.sql # Export to SQL file
pgread -d /path/to/data/ # Specify data directory
pgread -d /path/to/data/ -db mydb # Specific database
pgread -d /path/to/data/ -t password # Filter tables
pgread -d /path/to/data/ -list # Schema only
pgread -f /path/to/1262 # Parse single file# Security / Forensics
pgread -passwords all # Extract ALL password hashes
pgread -passwords postgres # Extract specific user's hash
pgread -secrets auto # Auto-detect secrets (API keys, etc)
pgread -search "password|secret"# Search with regex
pgread -deleted # Include deleted rows (forensics)
pgread -wal # WAL transaction summary
pgread -detect # Show detected PostgreSQL paths# Low-Level / Forensics
pgread -control # pg_control file (version, state, LSN)
pgread -checksum # Verify page checksums (corruption)
pgread -dropped # Show dropped columns (recoverable)
pgread -sequences all # List all sequences with values
pgread -relmap global # Show pg_filenode.map (OID→filenode)
pgread -f /path/to/file -R 0:10 # Read specific block range
pgread -f /path/to/index -index # Parse index file (BTree/GIN/GiST/Hash)
pgread -encoding GBK -sql # Output in GBK encoding (auto-detects DB encoding)

Password Extraction

$ pgread -passwords all
PostgreSQL Password Hashes:
===========================
postgres:SCRAM-SHA-256$4096:salt$hash:proof [SUPERUSER] [LOGIN]
admin:SCRAM-SHA-256$4096:salt$hash:proof [LOGIN]

Secret Detection (Powered by Trufflehog)

Uses trufflehog's 700+ detectors:

$ pgread -secrets auto
[
{
"detector": "Stripe",
"database": "postgres",
"table": "api_keys",
"column": "value",
"raw": "sk_live_51Hx...",
"extra_data": {
"rotation_guide": "https://howtorotate.com/docs/tutorials/stripe/"
}
}
]

Detects: Stripe, AWS, GitHub, GitLab, Slack, SendGrid, Doppler, DigitalOcean, Heroku, and 700+ more.

WAL Analysis

$ pgread -wal
{
"segment_count": 1,
"record_count": 24574,
"pg_version": "16",
"operations": {
"INSERT": 4440,
"DELETE": 106,
"UPDATE": 379,
"COMMIT": 738,
...
},
"transactions": [...]
}

pg_control Parsing

$ pgread -control
{
"pg_control_version": 1300,
"catalog_version_no": 202307071,
"system_identifier": 7123456789012345678,
"state": 6,
"state_string": "IN_PRODUCTION",
"checkpoint_lsn": 4294967376,
"checkpoint_lsn_str": "0/100000050",
"pg_version_major": 16,
"data_checksums_enabled": true,
...
}

Checksum Verification

$ pgread -checksum
{
"data_dir": "/var/lib/postgresql/data",
"checksums_enabled": true,
"total_files": 42,
"total_blocks": 1024,
"valid_blocks": 1024,
"invalid_blocks": 0
}

Detects page corruption before PostgreSQL does!

Index Parsing

$ pgread -f /var/lib/postgresql/data/base/16384/16385 -index
{
"type": 1,
"type_string": "btree",
"total_pages": 5,
"root_page": 1,
"levels": 2,
"meta": {
"magic": 340322,
"version": 4,
"root": 1,
"level": 2
},
"pages": [...]
}

Supports: BTree, GIN, GiST, Hash, SP-GiST

Dropped Columns Recovery

$ pgread -dropped
[
{
"database": "mydb",
"dropped_count": 2,
"columns": [
{
"rel_oid": 16384,
"table_name": "users",
"attnum": 3,
"dropped_name": "........pg.dropped.3........",
"type_oid": 25,
"type_name": "text"
}
]
}
]

Recover data from columns that were ALTER TABLE DROP COLUMN!

Sequence Parsing

$ pgread -sequences mydb
[
{
"name": "users_id_seq",
"oid": 16396,
"filenode": 16396,
"last_value": 42,
"start_value": 1,
"increment_by": 1,
"max_value": 9223372036854775807,
"min_value": 1,
"is_cycled": false,
"is_called": true
}
]

pg_filenode.map Parsing

$ pgread -relmap global
{
"magic": 5842711,
"num_mappings": 50,
"mappings": [
{"oid": 1262, "filenode": 1262},
{"oid": 1260, "filenode": 1260},
...
]
}

Maps system catalog OIDs to their physical filenodes.

Block Range Selection

$ pgread -f /path/to/heap -R 0:5
[
{
"block_number": 0,
"lsn": "0/19921E0",
"checksum": 0,
"lower": 212,
"upper": 7744,
"page_size": 8192,
"item_count": 47,
"free_space": 7532
},
...
]

Read specific blocks: 0:10 (blocks 0-10), 5: (from 5), :20 (up to 20), 5 (block 5 only).

Binary Block Dump

$ pgread -f /path/to/heap -b -R 0
Block 0 (offset 0x00000000):
00000000 00 00 00 00 40 2e 4f 01 00 00 01 00 30 00 20 1d |....@.O.....0. .|
00000010 00 20 04 20 00 00 00 00 05 00 01 00 06 00 01 00 |. . ............|
...

Raw hex dump like xxd or hexdump -C. Useful for low-level forensics.

Multi-Segment Files

PostgreSQL splits large tables into 1GB segments. pgread handles this:

# Read from specific segment
$ pgread -f /path/to/16384.2 -n 2 -R 0:10
# Custom segment size (e.g., 128MB for some configs)
$ pgread -f /path/to/file -s 134217728 -R 0:100

TOAST Verbose

$ pgread -f /path/to/toast_table -toast-verbose
{
"toast_rel_id": 16385,
"total_chunks": 150,
"unique_values": 42,
"total_size": 1048576,
"average_chunk_size": 6990.5,
"max_chunks_per_value": 12,
"chunk_distribution": {"1": 20, "2": 15, "5": 5, "12": 2}
}

Table Output (psql-style)

$ pgread -d /path/to/data -db mydb -t users -table
mydb.users (3 rows)
email | password_hash | is_admin
--------------------+-----------------------------------------------+---------
admin@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|true
alice@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
bob@example.com |$argon2id$v=19$m=19456,t=2,p=1$salt$hash|false
(3 rows)

SQL/CSV Export

# Export entire database
pgread -sql -db mydb > mydb_backup.sql
psql -d newdb < mydb_backup.sql
# Export to CSV
pgread -csv -db mydb > mydb.csv

Library

import"github.com/Chocapikk/pgread/pgdump"// Auto-detect and dump ALL PostgreSQL instancesresults, _:=pgdump.DumpAll(nil)
// Or specify a pathresult, _:=pgdump.DumpDataDir("/var/lib/postgresql/data", nil)
// With optionsresult, _:=pgdump.DumpDataDir("/path/to/data", &pgdump.Options{
DatabaseFilter: "mydb",
TableFilter: "password",
})
// Custom file reader (arbitrary file read, SSRF, backups, etc.)pgdump.DumpDatabaseFromFiles(classData, attrData, func(fnuint32) ([]byte, error) {
returnhttpClient.Get(fmt.Sprintf("/base/%d/%d", dbOID, fn))
}, nil)
// Export to SQLresult, _:=pgdump.DumpDataDir("/path/to/data", nil)
result.ToSQL(os.Stdout) // or any io.Writer

Auto-Detection

// Find first PostgreSQL data directorydataDir:=pgdump.DetectDataDir()
// Find ALL PostgreSQL data directoriesdataDirs:=pgdump.DetectAllDataDirs()

Low-Level API

// Parse system catalogsdatabases:=pgdump.ParsePGDatabase(data) // []DatabaseInfotables:=pgdump.ParsePGClass(data) // map[filenode]TableInfocolumns:=pgdump.ParsePGAttribute(data,0) // map[oid][]AttrInfo// Decode table datarows:=pgdump.ReadRows(tableData, schema, true)
// Raw tuple accesstuples:=pgdump.ReadTuples(data, true)
row:=pgdump.DecodeTuple(tuple, columns)
// pg_control parsingcontrol, _:=pgdump.ReadControlFile(dataDir)
fmt.Printf("PG Version: %d, State: %s\n", control.PGVersionMajor, control.StateString)
// Checksum verificationresult, _:=pgdump.VerifyDataDirChecksums(dataDir)
fmt.Printf("Valid: %d, Invalid: %d\n", result.ValidBlocks, result.InvalidBlocks)
// Index parsingindexInfo, _:=pgdump.ParseIndexFile(data)
fmt.Printf("Type: %s, Root: %d\n", indexInfo.TypeString, indexInfo.RootPage)
// Dropped columnsdropped, _:=pgdump.FindDroppedColumns(dataDir, "mydb")
for_, col:=rangedropped.Columns {
fmt.Printf("Dropped: %s.%d (%s)\n", col.TableName, col.AttNum, col.TypeName)
}

Supported Types

Numeric:boolint2int4int8float4float8numericmoney

Text:textvarcharcharbpcharnamebytea

Date/Time:datetimetimetztimestamptimestamptzinterval

Network:inetcidrmacaddrmacaddr8

Geometric:pointlinelsegboxcirclepathpolygon

Structured:jsonjsonbjsonpathxmluuid

Range:int4rangeint8rangenumrangedaterangetsrangetstzrange

Text Search:tsvectortsquery

Other:oidtidxidcidpg_lsnbitvarbit + arrays of all above

Build

go build
GOOS=windows go build -o pgread.exe
GOOS=darwin GOARCH=arm64 go build -o pgread-macos

Encoding Support

pgread auto-detects the database encoding from pg_database and converts to UTF-8 by default. Use -encoding to output in a specific charset:

pgread -sql -encoding GBK > dump.sql # Output in GBK
pgread -sql # Default: UTF-8

Supported: UTF-8, GBK, GB18030, BIG5, SJIS, EUC-JP, EUC-KR, EUC-CN, LATIN1-5, WIN1250-1258, KOI8-R, KOI8-U, ISO-8859-5/6/7/8.

Known Limitations

  • TOAST & Compression: Large values stored in TOAST tables are automatically resolved, including after VACUUM FULL. Supports PGLZ and LZ4 compression, both inline (small compressed values kept in the main heap) and external (chunked in TOAST tables). Works with PostgreSQL 12-17.
  • Encrypted data: Application-level encryption is returned as-is (ciphertext). pgread extracts what PostgreSQL stores.
  • In-flight data: Recently written data still in shared buffers may not be on disk yet. Run CHECKPOINT first if possible.

Related

License

WTFPL - Do What The Fuck You Want To Public License

About

Read PostgreSQL data files without credentials - forensics, data recovery, and security research tool

Topics

Resources

Stars

49 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages