DUMPDB: A LDMUD OBJ_DUMP -> SQLite3 DB Tool

About

The LDMUD MUD driver supports dumping a formatted text file with information about every object loaded at the time of the dump. This is an invaluable tool for finding object leaks and understanding your game's memory usage.

The format of OBJ_DUMP can be challenging to work with using shell tools as some fields are optional and may be omitted. Having the contents parsed and represented in a small database makes the data easier to explore. Use dumpdb to answer questions like:

• What is the most cloned object in memory?
• What is the largest memory usage by base file?
• What object has the largest tick count?
• What base file has the largest tick count?

Usage

1. Install dumpdb.
2. Install the sqlite3 command line tools for your OS:

# For example...
apt-get install sqlite3

3. In-game, collect an OBJ_DUMP by calling:

efun::dump_driver_info(DDI_OBJECTS);

4. Out of game, run dumpdb on your OBJ_DUMP file:

dumpdb /path/to/my/game/lib/OBJ_DUMP

Processing a 5.5MB OBJ_DUMP with ~47,000 lines takes approximately 6s and creates a 6.5MB SQLite database.

5. Explore the db with sqlite3 OBJ_DUMP.sqlite:

sqlite3 OBJ_DUMP.sqlite

.tables
.schema obj_dump
SELECTCOUNT(id) FROM obj_dump;

Tested with OBJ_DUMP's collected from LDMud 3.6.4 and 3.5.4.

Installation (Linux x86_64)

If you're on 64bit Linux you can download a pre-built executable. Other platforms should build from source.

1. Download a pre-built release.
2. Extract the .tar.gz.

tar -xvf dumpdb_*_linux_amd64.tar.gz

3. Run the dumpdb executable.

./dumpdb /path/to/my/game/lib/OBJ_DUMP

Installation from Source

• Set up Go 1.16+ by following the Go install docs.
• You will also need a C compiler to support the CGo sqlite dependency. E.g.

apt-get install gcc

• Clone the source code from git and change to the directory:

git clone https://github.com/dune-mud/dumpdb.git &&cd dumpdb

• Build and install

go install -ldflags "-X main.commit=$(git rev-parse HEAD)" ./...

• Run the dumpdb executable from your Go bin.

$GOPATH/bin/dumpdb /path/to/my/game/lib/OBJ_DUMP

Database Schema

The obj_dump table that is created in the SQLite3 database has the following schema:

CREATETABLEIF NOT EXISTS obj_dump (
id INTEGERPRIMARY KEY AUTOINCREMENT,
name TEXTNOT NULL,
base_file TEXTNOT NULL,
size INTEGERNOT NULL,
full_size INTEGERNOT NULL,
refs INTEGERNOT NULL DEFAULT 0,
hb INTEGERNOT NULL DEFAULT 0,
environment TEXTNOT NULL DEFAULT "",
ticks INTEGERNOT NULL DEFAULT 0,
swap_status TEXTNOT NULL DEFAULT "",
created TEXTNOT NULL
);

OBJ_DUMP format

The OBJ_DUMP format is described in man dump_driver_info:

For every object, a line is written into the file with the following information in the given order:

• object name
• size in memory, shared data counted only once
• size in memory if data wouldn't be shared
• number of references
• 'HB' if the object has a heartbeat, nothing if not.
• the name of the environment, or '--' if the object has no environment
• in parentheses the number of execution ticks spent in this object
• the swap status: nothing if not swapped, 'PROG SWAPPED' if only the program is swapped, 'VAR SWAPPED' if only the variables are swapped, 'SWAPPED' if both program and variables are swapped
• the time the object was created

Handy Queries

• Total number of objects.

SELECTCOUNT(id) FROM obj_dump;

• Total number of base files (E.g. name with the "#xxxxxx" clone reference removed).

SELECTCOUNT(DISTINCT base_file)
FROM obj_dump;

• Top 25 largest users of memory by object.

SELECT name, size FROM obj_dump ORDER BY size DESCLIMIT25;

• Top 25 largest users of memory by base file.

SELECT base_file, SUM(size) AS usage
FROM obj_dump
GROUP BY base_file
ORDER BY usage DESCLIMIT25;

• Top 25 most cloned base files.

SELECT base_file, COUNT(id) AS clones FROM obj_dump
GROUP BY base_file
ORDER BY clones DESCLIMIT25;

• Top 25 largest tick counts by object:

SELECT name, ticks
FROM obj_dump
ORDER BY ticks DESCLIMIT25;

• Top 25 largest tick counts by base file:

SELECT base_file, SUM(ticks) AS usage
FROM obj_dump
GROUP BY base_file
ORDER BY usage DESCLIMIT25;

• Top 25 objects by reference count:

SELECT name, refs
FROM obj_dump
ORDER BY refs DESCLIMIT25;

• Top 25 base files by reference count:

SELECT base_file, SUM(refs) AS refs
FROM obj_dump
GROUP BY base_file
ORDER BY refs DESCLIMIT25;

• Number of clones from a particular directory:

SELECTCOUNT(id) FROM obj_dump WHERE name LIKE'd/Ix/paradox/spire/%';