Skip to content

Repository files navigation

Census API AI Agent

An MCP (Model Context Protocol) server that powers an AI agent for working with US Census Bureau datasets. Enables users to discover relevant variables, construct API requests, and retrieve geographic reference data through natural language queries.

Motivation

US Census datasets span thousands of variables across hundreds of surveys — making variable discovery and API construction a significant bottleneck in social science research workflows. This project removes that friction by layering an AI agent on top of the Census API, combining structured SQL lookups with semantic RAG-based search.

Architecture

User / Claude Desktop
│
▼
FastMCP Server (MCP tools)
│
┌────┴────────────────────┐
│ │
SQLite (geographic + ChromaDB (RAG)
catalog reference) variable search

MCP Tools exposed to the AI agent:

ToolDescription
setup_api_keyValidates and persists a Census Bureau API key
check_api_keyConfirms whether a key is configured
get_api_key_instructionsReturns Census Bureau signup guidance
query_state_countyConverts state/county names → FIPS codes + API query fragments
discovery_variablesRAG-powered variable discovery from natural language context
fetch_census_dataFetches Census API data for specified variable IDs

Tech Stack

LayerTechnology
Agent ProtocolFastMCP (MCP server framework)
Vector StoreChromaDB + sentence-transformers
Relational DBSQLite + SQLAlchemy ORM
Data Wranglingpandas
HTTP Clienthttpx (async Census API calls)
RuntimePython 3.12

Key Components

app/core/variable.py — Fetches Census variable catalogs via the Data Discovery API, parses variable metadata, and builds semantically rich ChromaDB chunks (e.g., "age, categorized by sex, specifically male").

app/core/geographic.py — Bulk-loads state and county FIPS reference data from Census CSVs into SQLite and provides FIPS lookup for query construction.

app/core/storage.py — Orchestrates database initialization: creates the SQLAlchemy schema, loads catalog data, and upserts variable embeddings into ChromaDB.

app/mcp/server.py — Defines all FastMCP tool handlers, bridging AI agent requests to the underlying SQL and RAG backends.

Database Schema

ref_states (STATEFP, STUSAB, STATENAME_FULL)
ref_counties (STATEFP FK, COUNTYFP, COUNTYNAME)
ref_catalog (vintage, route, frequency, variables_url, title, description)
ref_variable_groups (catalog_id FK, group_code, label)
ref_variables (group_id FK, variable, label, concept, predicate_type)

ChromaDB collection census_variables stores variable documents with hierarchical semantic labels for similarity search.

Getting Started

Prerequisites: Conda, Python 3.12, a Census API key

# 1. Activate environment
conda activate ai_agent
# 2. Install dependencies
pip install -e .# 3. Run the MCP server
python main.py

Then configure the server in Claude Desktop's claude_desktop_config.json and use the setup_api_key tool to store your Census Bureau API key.

Use Cases

  • Sociologists and economists exploring ACS, Decennial Census, or CPS variables
  • Researchers who need to quickly identify variable codes without reading full documentation
  • Data pipelines that need programmatic Census API URL construction
  • Anyone integrating Census data into an AI-assisted analysis workflow

About

Census Dataset AI Agent

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages