A SQLite extension for making HTTP requests purely in SQL.
- Create GET, POST, and other HTTP requests, like
curl,wget, andfetch - Download response bodies, header, status codes, timing info
- Set rate limits, timeouts
.load ./http0
select http_get_body('https://text.npr.org/');
/*<!DOCTYPE html><html lang="en"><head> <title>NPR : National Public Radio</title> ....*/Query for all custom headers in an endpoint.
select name, value
from http_headers_each(
http_get_headers('https://api.github.com/')
)
where name like'X-%';
/*┌────────────────────────┬────────────────────────────────────┐│ name │ value │├────────────────────────┼────────────────────────────────────┤│ X-Ratelimit-Limit │ 60 ││ X-Ratelimit-Used │ 8 ││ X-Content-Type-Options │ nosniff ││ X-Github-Media-Type │ github.v3; format=json ││ X-Github-Request-Id │ CCCA:5FDF:1014BC2:10965F9:62F3DE4E ││ X-Ratelimit-Remaining │ 52 ││ X-Ratelimit-Resource │ core ││ X-Frame-Options │ deny ││ X-Ratelimit-Reset │ 1660152798 ││ X-Xss-Protection │ 0 │└────────────────────────┴────────────────────────────────────┘*/Scrape data from a JSON endpoint.
select http_get_body('https://api.github.com/repos/sqlite/sqlite')
->>'$.description'as description;
/*┌───────────────────────────────────────────────┐│ description │├───────────────────────────────────────────────┤│ Official Git mirror of the SQLite source tree │└───────────────────────────────────────────────┘*/Pass in specific headers in a request.
select
value
from json_each(
http_get_body(
'https://api.github.com/issues',
http_headers(
'Authorization', 'token ghp_16C7e42F292c6912E7710c8'
)
)
);
See docs.md for a full API reference.
| Language | Install | |
|---|---|---|
| Python | pip install sqlite-http |  |
| Datasette | datasette install datasette-sqlite-http |  |
| Node.js | npm install sqlite-http |  |
| Deno | deno.land/x/sqlite_http |  |
| Ruby | gem install sqlite-http |  |
| Github Release |  |
The Releases page contains pre-built binaries for Linux amd64, MacOS amd64 (no arm), and Windows.
If you want to use sqlite-http as a Runtime-loadable extension, Download the http0.dylib (for MacOS), http0.so (Linux), or http0.dll (Windows) file from a release and load it into your SQLite environment.
Note: The
0in the filename (http0.dylib/http0.so/http0.dll) denotes the major version ofsqlite-http. Currentlysqlite-httpis pre v1, so expect breaking changes in future versions.
For example, if you are using the SQLite CLI, you can load the library like so:
.load ./http0
select http_version();
-- v0.0.1Or in Python, using the builtin sqlite3 module:
importsqlite3con=sqlite3.connect(":memory:")
con.enable_load_extension(True)
con.load_extension("./http0")
print(con.execute("select http_version()").fetchone())
# ('v0.0.1',)Or in Node.js using better-sqlite3:
constDatabase=require("better-sqlite3");constdb=newDatabase(":memory:");db.loadExtension("./http0");console.log(db.prepare("select http_version()").get());// { 'http_version()': 'v0.0.1' }Or with Datasette, with the "no network" option to limit DDoS attacks:
datasette data.db --load-extension ./http0-no-net
- sqlite-html, for parsing and querying HTML using CSS selectors in SQLite (pairs great with this tool)
- pgsql-http, a similar yet very different HTTP library for POstgreSQL (didn't know about this before I started this, but interestingly enough came up with a very similar API)
- riyaz-ali/sqlite, the brilliant Go library that this library depends on
- nalgeon/sqlean, several pre-compiled handy SQLite functions, in C