Skip to content

Repository files navigation

Virtuals

Virtual Python collections over any storage.

Dict, list, set, indexed dict, tree — they look and feel like native Python, but they don't physically exist as in-memory collections. They're virtual: lazy views that compose data structure logic over flat tuple-key storage. Any backend that implements the storage protocol gets every data structure for free.

Like SQLAlchemy for Python collections. No SQL, no specific backend. Define your structure, plug in a store.

PyPI: virtuals-py | Import: virtuals

What It Does

fromvirtualsimportView, Containerfromvirtuals.storages.memimportInMemoryStoragefromvirtuals.codecsimportNoOpCodecstorage=InMemoryStorage(codec=NoOpCodec())
storage.open()
withstorage.transaction() astx:
users=DictView.open_root(tx)
users["alice"] = {"name": "Alice", "age": 30}
users["bob"] = {"name": "Bob", "age": 25}
# Navigate naturallyforuser_id, profileinusers.items():
print(f"{user_id}: {profile['name']}")
# Under the hood: flat KV pairs with tuple keys# ("users", "alice", "name") -> "Alice"# ("users", "alice", "age") -> 30

Three Layers

Layer 1: Storage

Generic tuple-key KV store with lexicographic ordering.

storage.put(("users", "alice", "name"), "Alice")
value=storage.get(("users", "alice", "name"))

Layer 2: Container

Hierarchy and parent-child relationships over flat keys.

container=Container(storage, ("users", "alice"))
container.create()
children=container.children() # ["name", "age"]

Layer 3: View

Data structure abstractions (dict, list, set) over containers.

users=DictView.open_root(tx)
users["alice"] ="data"

Installation

pip install virtuals-py
pip install virtuals-py[rocksdb] # with RocksDB backend

Features

  • Virtual collections: Work with dicts, lists, sets over any KV backend
  • Tuple keys: Natural hierarchical addressing with lexicographic ordering
  • Backend agnostic: Works with any ordered KV store (RocksDB, LMDB, in-memory)
  • Observable: Watch for changes at any level of the hierarchy
  • Transactional: Full ACID support when the backend provides it
  • Lazy: Nothing materializes until accessed

License

Apache-2.0

About

Virtual Python collections over any storage

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages