BranchesTags

Repository files navigation

What if creating a database would be as cheap as creating a Hashmap?

An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.

DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.

DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.

The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:

  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.

Latest version Build Status

;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}

Important! If you are using shadow-cljs, add

:compiler-options {:externs ["datascript/externs.js"]}

to your build (see #432#298#216)

Support us

Resources

Support:

Books:

Docs:

Posts:

Talks:

  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video

Projects using DataScript:

Related projects:

  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.

Demo applications:

Usage examples

For more examples, see our acceptance test suite.

(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]

Using from vanilla JS

DataScript can be used from any JS engine without additional dependencies:

<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>

or as a CommonJS module (npm page):

npm install datascript

vards=require('datascript');

or as a RequireJS module:

require(['datascript'],function(ds){ ... });

Queries:

  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays

Entities:

  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)

Transactions:

  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with

Transaction reports:

  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence

Check out test/js/tests.js for usage examples.

Project status

Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.

The following features are supported:

  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)

Query engine features:

  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications

Interface differences:

  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.

Differences from Datomic

  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies

Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.

Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.

Contributing

Testing

Setup

npm install ws

Running the tests

clj -M:test -m kaocha.runner

Watching tests:

./script/watch.sh

Benchmarking and Datomic compatibility

datomic-free is a dependency not available on Clojars or Maven Central.

  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install

Run compatibility checks:

clj -M:datomic

Benchmark:

cd bench
./bench.clj

License

Copyright © 2014–2024 Nikita Prokopov

Licensed under Eclipse Public License (see LICENSE).

About

Immutable database and Datalog query engine for Clojure, ClojureScript and JS

Resources

Readme
License
Activity
Custom properties

Stars

8 stars

Watchers

2 watching

Forks

2 forks
Report repository

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
What if creating a database would be as cheap as creating a Hashmap?
An immutable in-memory database and Datalog query engine in Clojure and ClojureScript.
DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.
DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.
The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:
  • Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
  • Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
  • Datalog query engine to answer non-trivial questions about current app state.
  • Structured format to track data coming in and out of DB. Datalog queries can be run against it too.
Latest version Build Status
;; lein
[datascript "1.7.3"]
;; deps.edn
datascript/datascript {:mvn/version"1.7.3"}
Important! If you are using shadow-cljs, add
:compiler-options {:externs ["datascript/externs.js"]}
to your build (see #432#298#216)
Support us
Resources
Support:
Books:
Docs:
Posts:
Talks:
  • “Frontend with Joy” talk (FPConf, August 2015): video in Russian
  • “Programming Web UI with Database in a Browser” talk (PolyConf, July 2015): slides, video
  • “DataScript for Web Development” talk (Clojure eXchange, Dec 2014): slides, video
  • “Building ToDo list with DataScript” webinar (ClojureScript NYC, Dec 2014): video, app
  • DataScript hangout (May 2014, in Russian): video
Projects using DataScript:
Related projects:
  • DataScript-Transit, transit serialization for database and datoms
  • DataScript SQL Storages, durable storage implementations for popular SQL databases
  • Posh, lib that lets you use a single DataScript db to store Reagent app state
  • re-posh, use re-frame with DataScript storage
  • DataScript-mori, DataScript & Mori wrapper for use from JS
  • DatSync, Datomic ↔︎ DataScript syncing/replication utilities
  • Intension, lib to convert associative structures to in-memory databases for querying them
  • Datamaps, lib designed to leverage datalog queries to query arbitrary maps.
Demo applications:
Usage examples
For more examples, see our acceptance test suite.
(require '[datascript.core :as d])
;; Implicit join, multi-valued attribute
(let [schema {:aka {:db/cardinality:db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [ { :db/id-1:name"Maksim":age45:aka ["Max Otto von Stierlitz", "Jack Ryan"] } ])
(d/q '[ :find ?n ?a
:where [?e :aka"Max Otto von Stierlitz"]
[?e :name ?n]
[?e :age ?a] ]
@conn))
;; => #{ ["Maksim" 45] };; Destructuring, function call, predicate call, query over collection
(d/q '[ :find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)] ]
{ :a [17], :b [24] }
range)
;; => #{ [:a 2] [:a 4] [:a 6] [:b 2] };; Recursive rule
(d/q '[ :find ?u1 ?u2
:in $ %
:where (follows ?u1 ?u2) ]
[ [1:follows2]
[2:follows3]
[3:follows4] ]
'[ [(follows ?e1 ?e2)
[?e1 :follows ?e2]]
[(follows ?e1 ?e2)
[?e1 :follows ?t]
(follows ?t ?e2)] ])
;; => #{ [1 2] [1 3] [1 4];; [2 3] [2 4];; [3 4] };; Aggregates
(d/q '[ :find ?color (max ?amount ?x) (min ?amount ?x)
:in [[?color ?x]] ?amount ]
[[:red10] [:red20] [:red30] [:red40] [:red50]
[:blue7] [:blue8]]
3)
;; => [[:red [30 40 50] [10 20 30]];; [:blue [7 8] [7 8]]]
Using from vanilla JS
DataScript can be used from any JS engine without additional dependencies:
<scriptsrc="https://github.com/tonsky/datascript/releases/download/1.7.3/datascript-1.7.3.min.js"></script>
or as a CommonJS module (npm page):
npm install datascript

vards=require('datascript');
or as a RequireJS module:
require(['datascript'],function(ds){ ... });
Queries:
  • Query and rules should be EDN passed as strings
  • Results of q are returned as regular JS arrays
Entities:
  • Entities returned by entity call are lazy as in Clojure
  • Use e.get("prop"), e.get(":db/id"), e.db to access entity properties
  • Entities implement ECMAScript 6 Map interface (has/get/keys/...)
Transactions:
  • Use strings such as ":db/id", ":db/add", etc. instead of db-namespaced keywords
  • Use regular JS arrays and objects to pass data to transact and db_with
Transaction reports:
  • report.tempids has string keys ("-1" for entity tempid -1), use resolve_tempid to set up a correspondence
Check out test/js/tests.js for usage examples.
Project status
Stable. Most of the features done, expecting non-breaking API additions and performance optimizations. No docs at the moment, use examples & Datomic documentation.
The following features are supported:
  • Database as a value: each DB is an immutable value. New DBs are created on top of old ones, but old ones stay perfectly valid too
  • Triple store model
  • EAVT, AEVT and AVET indexes
  • Multi-valued attributes via :db/cardinality :db.cardinality/many
  • Lazy entities and :db/valueType :db.type/ref auto-expansion
  • Database “mutations” via transact!
  • Callback-based analogue to txReportQueue via listen!
  • Direct index lookup and iteration via datoms and seek-datoms
  • Filtered databases via filter
  • Lookup refs
  • Unique constraints, upsert
  • Pull API (thx David Thomas Hume)
Query engine features:
  • Implicit joins
  • Query over DB or regular collections
  • Parameterized queries via :in clause
  • Tuple, collection, relation binding forms in :in clause
  • Query over multiple DB/collections
  • Predicates and user functions in query
  • Negation and disjunction
  • Rules, recursive rules
  • Aggregates
  • Find specifications
Interface differences:
  • Conn is just an atom storing last DB value, use @conn instead of (d/db conn)
  • Instead of #db/id[:db.part/user -100] just use -100 in place of :db/id or entity id
  • Transactor functions can be called as [:db.fn/call f args] where f is a function reference and will take db as first argument (thx @thegeez)
  • In ClojureScript, custom query functions and aggregates should be passed as source instead of being referenced by symbol (due to lack of resolve in CLJS)
  • Custom aggregate functions are called via aggregate keyword: :find (aggregate ?myfn ?e) :in $ ?myfn
  • Additional :db.fn/retractAttribute shortcut
  • Transactions are not annotated by default with :db/txInstant
  • When “transaction function” is called, the db that this function receive is a “partial db” relative to it's position in transaction.
Differences from Datomic
  • DataScript is built totally from scratch and is not related by any means to the popular Clojure database Datomic
  • Runs in a browser and/or in a JVM
  • Simplified schema, not queryable
  • Attributes do not have to be declared in advance. Put them to schema only when you need special behaviour from them
  • Any type can be used for values
  • No :db/ident for attributes, keywords are literally attribute values, no integer id behind them
  • No schema migrations
  • No full-text search, no partitions
  • No external dependencies
Aimed at interactive, long-living browser applications, DataScript DBs operate in constant space. If you do not add new entities, just update existing ones, or clean up database from time to time, memory consumption will be limited. This is unlike Datomic which keeps history of all changes, thus grows monotonically. DataScript does not track history by default, but you can do it via your own code if needed.
Some of the features are omitted intentionally. Different apps have different needs in storing/transfering/keeping track of DB state. DataScript is a foundation to build exactly the right storage solution for your needs without selling too much “vision”.
Contributing
Testing
Setup
npm install ws

Running the tests
clj -M:test -m kaocha.runner

Watching tests:
./script/watch.sh

Benchmarking and Datomic compatibility
datomic-free is a dependency not available on Clojars or Maven Central.
  1. Download datomic-free from https://my.datomic.com/downloads/free
  2. Unzip it
  3. Inside the unzipped folder run ./bin/maven-install
Run compatibility checks:
clj -M:datomic

Benchmark:
cd bench
./bench.clj

License
Copyright © 2014–2024 Nikita Prokopov
Licensed under Eclipse Public License (see LICENSE).
About
Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Resources
Readme
License
Activity
Custom properties
Stars
8 stars
Watchers
2 watching
Forks
2 forks
Report repository
Releases
Packages
Contributors
Languages