Skip to content

Repository files navigation

OpenFOIA

Local-first investigation toolkit for journalists, researchers, and citizens.

Your data never leaves your machine. Works offline. Works everywhere.

One real government document → 45 entities, 166 relationships, extracted locally in seconds. No cloud. No API keys.

Entity relationship graph — 45 entities, 166 relationships extracted from a Palantir police surveillance contract pulled from DocumentCloud

Built-in document reader with entity highlighting — click any entity to jump to it in the source text, with a link to the original document


Install

With Claude Code (recommended)

If you use Claude Code, install the OpenFOIA plugin — you get the CLI plus a built-in copilot that knows every command:

/plugin marketplace add JordanCoin/openfoia
/plugin install openfoia@openfoia
/foia-install

The plugin ships a skill that teaches Claude how to run FOIA investigations, plus slash commands for the core loop: /foia-search, /foia-investigate, /foia-graph, /foia-install. The last one bootstraps the CLI on your machine.

Shell install (no Claude Code needed)

curl -fsSL https://raw.githubusercontent.com/JordanCoin/openfoia/main/install.sh | bash

Works on Linux, macOS, and Windows (WSL).

New to OpenFOIA? Run openfoia guide after installing.

What It Does

File FOIA requests. Analyze documents. Extract entities. Build relationship graphs. Cross-reference against public databases. All locally, all offline, all purgeable.

openfoia init # 53 federal agencies pre-loaded
openfoia records search "EPA water" --source muckrock # search 46k+ completed FOIA requests
openfoia records search "Palantir" --source documentcloud # search 10M+ public documents
openfoia records fetch 3411897 --source documentcloud # pull full text into local database
openfoia records download 68490 --ingest # download + ingest response documents
openfoia analyze extract <doc-id># extract people, orgs, money, dates
openfoia analyze graph --name investigation --view # interactive graph + document reader
openfoia crossref # check entities against 6 public databases

Features

PDF text extractionCompiled 1.1MB binary. 99.5% accuracy across 480K chars. Zero dependencies.
Entity extraction4-tier: LLM → GLiNER → spaCy → Regex. 100% recall with a 2GB local model
Relationship graphsInteractive graph with built-in document reader. Click entities to read source text with highlights
Cross-referenceMuckRock, DocumentCloud, OpenCorporates, SEC EDGAR, OpenSanctions, ICIJ Offshore Leaks
File requestsEmail, fax (Twilio), physical mail (Lob) to 53 federal agencies
Data interchangeImport/export FollowTheMoney format (Aleph, OpenAleph, OpenSanctions)
Encrypted storageSQLCipher AES-256. Decoy profile mode
Forensic purge3-pass overwrite, shell history scrub, free space fill
Portable modeopenfoia portable — everything stays on the USB, nothing on the host
Metadata strippingStrips EXIF, PDF author, DOCX revision history on file and web-UI ingest. Email attachments and archived web pages are stored as received — see THREAT_MODEL.md

PDF Extraction Engine

A compiled 1.1MB binary extracts text directly from PDF structures — no OCR needed for born-digital documents. Uses a multi-level resolution cascade to handle broken fonts, missing encodings, and government form PDFs that other tools can't read.

99.5% accuracy across 480K characters in 50 test PDFs. 100% on every FOIA document tested. Zero runtime dependencies. Falls back to OCR (Tesseract) for scanned documents.

OpenFOIA calls the extractor with the large-document runtime profile by default (--profile large). To control concurrency per process, set:

  • OPENFOIA_PDF_EXTRACT_MAX_WORKERS (for example 4 or 8)
  • Optional profile override: OPENFOIA_PDF_EXTRACT_PROFILE=standard|large

Data Sources

SourceWhatAuth
MuckRock46k+ completed FOIA requests with downloadable documentsFree
DocumentCloud10M+ public documents with pre-extracted textFree
OpenCorporatesGlobal company ownership, directors, filingsFree
SEC EDGARUS corporate filingsFree
OpenSanctionsSanctions lists, politically exposed personsFree (non-commercial)
ICIJ Offshore LeaksPanama/Pandora/Paradise Papers (local CSV)Free download

Documentation

GuideWhat
Journalist GuideFull walkthrough — start here if you're new
Tails OSRunning from Tails for maximum privacy
USB InstallEncrypted USB portable setup
Air-GappedOffline deployment overview

Privacy

Everything runs on 127.0.0.1. Token-authenticated. Encrypted at rest (optional). Forensically purgeable. Works offline. Open source.

For sensitive investigations:openfoia serve --tor

Portable mode:openfoia portable — data stays on the USB, nothing touches the host.

When you're done:openfoia purge --secure --yes

Uninstall

curl -fsSL https://raw.githubusercontent.com/JordanCoin/openfoia/main/uninstall.sh | bash

Legal Context for Contributors

Writing this code is legal. The US DOJ stated in August 2025: "Merely writing code, without ill intent, is not a crime." Open source encryption is exempt from US export controls. The Tor Project, Signal, SecureDrop, and Tails set the precedent. More context.

Credits

Built by people who believe in freedom of information. Inspired by MuckRock, DocumentCloud, and the Reporters Committee for Freedom of the Press.

License

AGPL-3.0 — Keep it open.


"Democracy dies in darkness. FOIA is how we turn on the lights."

About

Local-first FOIA automation with AI-powered document analysis. Your data never leaves your machine. 🔒🇺🇸

Topics

Resources

Stars

7 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages