The Python reference implementation of JSONLT (JSON Lines Table), a data format for storing keyed records in append-only files. JSONLT builds on JSON Lines and optimizes for version control. Modifications append new lines rather than rewriting existing content, producing clean and meaningful diffs.
Note
This package is under active development. The API may change before the 1.0 release.
- Specification: formal definition of the JSONLT format
- Documentation: guides, tutorials, and API reference
- Conformance tests: language-agnostic test suite
pip install jsonlt-python
# Or
uv add jsonlt-pythonRequires Python 3.10 or later.
fromjsonltimportTable# Create a table with initial recordstable=Table.from_records(
"users.jsonlt",
[
{"id": "alice", "role": "admin", "email": "alice@example.com"},
{"id": "bob", "role": "user", "email": "bob@example.com"},
],
key="id",
)
# Read recordsuser=table.get("alice") # Returns the record or Noneexists=table.has("bob") # Returns True# Update a recordtable.put({"id": "alice", "role": "admin", "email": "alice@newdomain.com"})
# Delete records (appends a tombstone)table.delete("bob")
# Later, load the existing tabletable=Table.from_file("users.jsonlt")
# Iterate over all recordsforrecordintable.all():
print(record)The underlying file after these operations:
{"$jsonlt":{"key":"id","version":1}}
{"id":"alice","email":"alice@example.com","role":"admin"}
{"id":"bob","email":"bob@example.com","role":"user"}
{"id":"alice","email":"alice@newdomain.com","role":"admin"}
{"id":"bob","$deleted":true}JSONLT works well for configuration, metadata, and small-to-medium datasets where you want human-readable files that play nicely with Git. It's a good fit when you need keyed record storage but don't want the overhead of a database, and when you want to see exactly what changed in a pull request. The append-only design supports safe concurrent access from multiple processes.
JSONLT is not a database. For large datasets, high write throughput, or complex queries, consider SQLite or a proper database.
JSONLT supports multi-field compound keys for composite identifiers:
orders=Table.from_records(
"orders.jsonlt",
[
{"customer_id": "alice", "order_id": 1, "total": 99.99},
{"customer_id": "alice", "order_id": 2, "total": 149.99},
],
key=("customer_id", "order_id"),
)
order=orders.get(("alice", 1))Transactions provide snapshot isolation and atomic writes with conflict detection:
fromjsonltimportTable, ConflictErrortable=Table("counters.jsonlt", key="name")
withtable.transaction() astx:
counter=tx.get("visits")
new_count= (counter["count"] +1) ifcounterelse1tx.put({"name": "visits", "count": new_count})
# Commits automatically; rolls back on exception# Handle concurrent modification conflictstry:
withtable.transaction() astx:
tx.put({"name": "counter", "value": 42})
exceptConflictErrorase:
print(f"Conflict on key: {e.key}")Tables can be used like dictionaries:
# Get a record (raises KeyError if not found)user=table["alice"]
# Set a record (key in record must match)table["alice"] = {"id": "alice", "role": "admin"}
# Delete a recorddeltable["bob"]
# Check membershipif"alice"intable:
print("Found alice")
# Iterate over keysforkeyintable:
print(key)
# Get record countprint(len(table))Methods like pop(), setdefault(), and update() also work. The keys(), values(), and items() methods return sorted lists rather than views to maintain JSONLT's deterministic key ordering.
Tables support value-based equality comparison:
table1=Table("users.jsonlt", key="id")
table2=Table("users.jsonlt", key="id")
# Equal if same path, key specifier, and recordsiftable1==table2:
print("Tables have identical content")Two tables are equal when they have the same resolved path, key specifier, and record state. Transactions are equal when they reference the same parent table instance and have identical snapshot state.
Tables and transactions are mutable and therefore not hashable (cannot be used as dictionary keys or in sets).
# Find all records matching a predicateexpensive=table.find(lambdar: r.get("price", 0) >100)
# Find with limittop_3=table.find(lambdar: r.get("in_stock"), limit=3)
# Find the first matchfirst=table.find_one(lambdar: r.get("category") =="electronics")# Compact the file (removes tombstones and superseded records)table.compact()
# Clear all recordstable.clear()
# Force reload from disktable.reload()| Method | Description |
|---|---|
Table(path, key) | Open or create a table |
Table.from_records(path, records, key) | Create table with records |
Table.from_file(path) | Load existing table |
get(key) | Get a record by key, or None |
has(key) | Check if a key exists |
put(record) | Insert or update a record |
delete(key) | Delete a record |
all() | Iterate all records |
keys() | Iterate all keys |
items() | Iterate (key, record) pairs |
count() | Number of records |
find(predicate, limit=None) | Find matching records |
find_one(predicate) | Find first match |
transaction() | Start a transaction |
compact() | Remove historical entries |
clear() | Remove all records |
reload() | Reload from disk |
Tables support table[key], table[key] = record, del table[key], len(table), key in table, and for key in table.
| Method | Description |
|---|---|
get(key) | Get from snapshot |
has(key) | Check in snapshot |
put(record) | Buffer a write |
delete(key) | Buffer a deletion |
commit() | Write to disk |
abort() | Discard changes |
Transactions support the same dictionary-like access as tables.
All exceptions inherit from JSONLTError:
| Exception | Description |
|---|---|
ParseError | Invalid file format |
InvalidKeyError | Invalid or missing key |
FileError | I/O error |
LockError | Cannot get lock |
LimitError | Size limit exceeded |
TransactionError | Invalid transaction state |
ConflictError | Write-write conflict |
This library implements the JSONLT 1.0 Specification. It provides both Lenient and Strict Parser conformance profiles, and generates Strict-conformant output by default.
As the reference implementation, jsonlt-python passes the JSONLT conformance test suite. Implementers of other JSONLT libraries can use these tests to verify compatibility.
The JSONLT format draws from related work including BEADS, which uses JSONL for git-backed structured storage.
The development of this library involved AI language models, specifically Claude (Anthropic). AI tools contributed to drafting code, tests, and documentation. Human authors made all design decisions and final implementations, and they reviewed, edited, and validated AI-generated content. The authors take full responsibility for the correctness of this software.
MIT License. See LICENSE for details.