chDB is an embedded SQL OLAP Engine powered by ClickHouse 1 For more details: The birth of chDB
- In-process SQL OLAP Engine, powered by ClickHouse
- No need to install ClickHouse
- Minimized data copy from C++ to Python with python memoryview
- Input&Output support Parquet, CSV, JSON, Arrow, ORC and 60+more formats, samples
- Support Python DB API 2.0, example
Get started with chdb using our Installation and Usage Examples
Currently, chDB supports Python 3.8+ on macOS and Linux (x86_64 and ARM64).
pip install chdb
python3 -m chdb SQL [OutputFormat]
python3 -m chdb "SELECT 1,'abc'" PrettyThe following methods are available to access on-disk and in-memory data formats:
(Parquet, CSV, JSON, Arrow, ORC and 60+)
You can execute SQL and return desired format data.
importchdbres=chdb.query('select version()', 'Pretty'); print(res)# See more data type format in tests/format_output.pyres=chdb.query('select * from file("data.parquet", Parquet)', 'JSON'); print(res)
res=chdb.query('select * from file("data.csv", CSV)', 'CSV'); print(res)
print(f"SQL read {res.rows_read()} rows, {res.bytes_read()} bytes, elapsed {res.elapsed()} seconds")# See more in https://clickhouse.com/docs/en/interfaces/formatschdb.query('select * from file("data.parquet", Parquet)', 'Dataframe')(Pandas DataFrame, Parquet file/bytes, Arrow bytes)
importchdb.dataframeascdfimportpandasaspd# Join 2 DataFramesdf1=pd.DataFrame({'a': [1, 2, 3], 'b': ["one", "two", "three"]})
df2=pd.DataFrame({'c': [1, 2, 3], 'd': ["①", "②", "③"]})
ret_tbl=cdf.query(sql="select * from __tbl1__ t1 join __tbl2__ t2 on t1.a = t2.c",
tbl1=df1, tbl2=df2)
print(ret_tbl)
# Query on the DataFrame Tableprint(ret_tbl.query('select b, sum(a) from __table__ group by b'))fromchdbimportsessionaschs## Create DB, Table, View in temp session, auto cleanup when session is deleted.sess=chs.Session()
sess.query("CREATE DATABASE IF NOT EXISTS db_xxx ENGINE = Atomic")
sess.query("CREATE TABLE IF NOT EXISTS db_xxx.log_table_xxx (x String, y Int) ENGINE = Log;")
sess.query("INSERT INTO db_xxx.log_table_xxx VALUES ('a', 1), ('b', 3), ('c', 2), ('d', 5);")
sess.query(
"CREATE VIEW db_xxx.view_xxx AS SELECT * FROM db_xxx.log_table_xxx LIMIT 4;"
)
print("Select from view:\n")
print(sess.query("SELECT * FROM db_xxx.view_xxx", "Pretty"))see also: test_stateful.py.
importchdb.dbapiasdbapiprint("chdb driver version: {0}".format(dbapi.get_client_info()))
conn1=dbapi.connect()
cur1=conn1.cursor()
cur1.execute('select version()')
print("description: ", cur1.description)
print("data: ", cur1.fetchone())
cur1.close()
conn1.close()fromchdb.udfimportchdb_udffromchdbimportquery@chdb_udf()defsum_udf(lhs, rhs):
returnint(lhs) +int(rhs)
print(query("select sum_udf(12,22)"))see also: test_udf.py.
For more examples, see examples and tests.
- For chdb specific examples and documentation refer to doc.chdb.io
- For SQL syntax, please refer to ClickHouse SQL Reference
- Demo chDB at ClickHouse v23.7 livehouse! and Slides
Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated. There are something you can help:
- Help test and report bugs
- Help improve documentation
- Help improve code quality and performance
We welcome bindings for other languages, please refer to bindings for more details.
Apache 2.0, see LICENSE for more information.
chDB is mainly based on ClickHouse1 for trade mark and other reasons, I named it chDB.
- Discord: https://discord.gg/D2Daa2fM5K
- Email: auxtenwpc@gmail.com
- Twitter: @chdb
Footnotes
ClickHouse® is a trademark of ClickHouse Inc. All trademarks, service marks, and logos mentioned or depicted are the property of their respective owners. The use of any third-party trademarks, brand names, product names, and company names does not imply endorsement, affiliation, or association with the respective owners. ↩↩2