Problem
When users run /discover or manually add warehouse connections, the system creates connection entries even when the required Python driver isn't installed. This leads to a broken state where connections exist but every operation fails with generic errors like "DuckDB is not installed in this environment."
The error messages don't specify which Python environment to install into, making recovery difficult.
Desired Behavior
- DuckDB should always work out-of-box — it's the local development default and should be bundled or auto-installed during first use
- Connection validation before creation —
/discover and warehouse_add should verify driver availability before creating a connection entry - Auto-install option — when a driver is missing, offer to install it into the engine's venv automatically
- Clear install instructions — if auto-install isn't possible, show the exact pip command with the correct Python path
Implementation Notes
Key Files
packages/altimate-engine/ — Python engine and its venv where drivers need to be installedpackages/altimate-code/src/tool/warehouse-test.ts — warehouse connection testingpackages/altimate-code/src/tool/warehouse-add.ts — warehouse addition (should validate before saving)packages/altimate-code/src/bridge/client.ts — bridge calls that surface driver errors
Approach
Phase 1: Validate before creating connections
- Before
warehouse_add saves a connection, call a bridge method to check if the required driver is importable - If not importable, show:
Driver 'duckdb' not found. Install with: <exact pip command with venv path> /discover should skip creating entries for connections it can't actually use
Phase 2: Auto-install drivers
- Add a bridge method
driver_install(driver_name) that runs pip install in the engine's venv - Prompt user: "DuckDB driver not installed. Install now? [Y/n]"
- Support drivers: duckdb, psycopg2-binary, snowflake-connector-python, google-cloud-bigquery, databricks-sql-connector
Phase 3: Bundle DuckDB
- Add
duckdb as a default dependency in the engine's pyproject.toml - Ensure it's always available for local development without extra steps
Industry Patterns
- SQLMesh: Bundles DuckDB as zero-config default for local dev
- Meltano: Auto-installs plugins into dedicated venvs on first use (
--install flag, default on) - dbt: Clear error "No data platform adapters installed!" with exact install commands per adapter
Acceptance Criteria
Problem
When users run
/discoveror manually add warehouse connections, the system creates connection entries even when the required Python driver isn't installed. This leads to a broken state where connections exist but every operation fails with generic errors like "DuckDB is not installed in this environment."The error messages don't specify which Python environment to install into, making recovery difficult.
Desired Behavior
/discoverandwarehouse_addshould verify driver availability before creating a connection entryImplementation Notes
Key Files
packages/altimate-engine/— Python engine and its venv where drivers need to be installedpackages/altimate-code/src/tool/warehouse-test.ts— warehouse connection testingpackages/altimate-code/src/tool/warehouse-add.ts— warehouse addition (should validate before saving)packages/altimate-code/src/bridge/client.ts— bridge calls that surface driver errorsApproach
Phase 1: Validate before creating connections
warehouse_addsaves a connection, call a bridge method to check if the required driver is importableDriver 'duckdb' not found. Install with: <exact pip command with venv path>/discovershould skip creating entries for connections it can't actually usePhase 2: Auto-install drivers
driver_install(driver_name)that runs pip install in the engine's venvPhase 3: Bundle DuckDB
duckdbas a default dependency in the engine's pyproject.tomlIndustry Patterns
--installflag, default on)Acceptance Criteria
warehouse_addvalidates driver availability before saving/discoverdoesn't create connections that can't work