Datafiles: A file-based ORM for Python dataclasses

Datafiles is a bidirectional serialization library for Python dataclasses to synchronize objects to the filesystem using type annotations. It supports a variety of file formats with round-trip preservation of formatting and comments, where possible. Object changes are automatically saved to disk and only include the minimum data needed to restore each object.

Some common use cases include:

• Coercing user-editable files into the proper Python types
• Storing program configuration and state in version control
• Loading data fixtures for demonstration or testing purposes
• Synchronizing application state using file sharing services
• Prototyping data models agnostic of persistence backends

Watch my lightning talk for a demo of this in action!

Overview

Take an existing dataclass such as this example from the documentation:

fromdataclassesimportdataclass@dataclassclassInventoryItem:
"""Class for keeping track of an item in inventory."""name: strunit_price: floatquantity_on_hand: int=0deftotal_cost(self) ->float:
returnself.unit_price*self.quantity_on_hand

and decorate it with a directory pattern to synchronize instances:

fromdatafilesimportdatafile@datafile("inventory/items/{self.name}.yml")classInventoryItem:
...

Then, work with instances of the class as normal:

>>>item=InventoryItem("widget", 3)

# inventory/items/widget.ymlunit_price: 3.0

Changes to the object are automatically saved to the filesystem:

>>>item.quantity_on_hand+=100

# inventory/items/widget.ymlunit_price: 3.0quantity_on_hand: 100

Changes to the filesystem are automatically reflected in the object:

# inventory/items/widget.ymlunit_price: 2.5# <= manually changed from "3.0"quantity_on_hand: 100

>>>item.unit_price2.5

Objects can also be restored from the filesystem:

>>>fromdatafilesimportMissing>>>item=InventoryItem("widget", Missing)
>>>item.unit_price2.5>>>item.quantity_on_hand100

Installation

Install this library directly into an activated virtual environment:

$ pip install datafiles

or add it to your Poetry project:

$ poetry add datafiles

Documentation

To see additional synchronization and formatting options, please consult the full documentation.