feat: DB migrations and DB setup

[Ankita-Advitot]

Add DB models and standalone migrations layer under db/

Moves the schema (models + Alembic migrations) into a self-contained db/ package . Includes Case, Document, GoldenRecord, PipelineResult, ValidationResult models, the initial migration creating all 5 tables (with pgvector extension), EncryptedString column type, and db/config.py/db/database.py for standalone DB access. docker-compose.yml updated to pgvector/pgvector:pg16; pyproject.toml gets pgvector + cryptography.

[Copilot]

Pull request overview

This PR introduces a standalone db/ package for the lending POC, containing SQLAlchemy models and Alembic migrations to manage the Postgres schema (including pgvector support) independently of the application layer.

Changes:

• Added SQLAlchemy models for Case, Document, GoldenRecord, PipelineResult, and ValidationResult, plus shared enums and an EncryptedString column type.
• Added a self-contained Alembic setup under db/migrations/ with versioned migrations to create all tables and required enum types/pgvector extension.
• Updated local dev infra and dependencies (pgvector-enabled Postgres image, new Python deps).

Reviewed changes

Copilot reviewed 20 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
lending-poc/pyproject.toml	Adds pgvector and cryptography dependencies required by the DB layer.
lending-poc/docker-compose.yml	Switches DB image to pgvector-enabled Postgres and changes host port mapping.
lending-poc/db/models/case.py	Adds Case model and relationships to other DB entities.
lending-poc/db/models/document.py	Adds Document model with doc type enum and extracted fields storage.
lending-poc/db/models/golden_record.py	Adds GoldenRecord model including a pgvector embedding column and encrypted fields.
lending-poc/db/models/pipeline_result.py	Adds PipelineResult model and review status enum.
lending-poc/db/models/validation_result.py	Adds ValidationResult model tied to cases/documents with evidence JSONB.
lending-poc/db/models/types.py	Adds EncryptedString TypeDecorator for encrypting sensitive string columns.
lending-poc/db/models/enums.py	Defines enums shared across DB schema and pipeline logic (doc/check/decision).
lending-poc/db/models/init.py	Exposes DB model imports for convenient registration and access.
lending-poc/db/migrations/env.py	Configures Alembic to run against the new db/ package metadata.
lending-poc/db/migrations/script.py.mako	Provides the Alembic revision template under the new migrations layout.
lending-poc/db/migrations/versions/0001_add_cases.py	Creates cases table and installs vector extension.
lending-poc/db/migrations/versions/0002_add_documents.py	Creates documents table and doc_type enum.
lending-poc/db/migrations/versions/0003_add_golden_records.py	Creates golden_records table including vector column and encrypted columns.
lending-poc/db/migrations/versions/0004_add_pipeline_results.py	Creates pipeline_results table and related enums.
lending-poc/db/migrations/versions/0005_add_validation_results.py	Creates validation_results table and related enum.
lending-poc/db/database.py	Adds standalone async engine/session setup and Base for the DB package.
lending-poc/db/config.py	Adds DB-layer settings (DATABASE_URL, ENCRYPTION_KEY, DEBUG).
lending-poc/db/alembic.ini	Points Alembic at the new db/migrations location and adjusts sys.path.
lending-poc/db/init.py	Initializes the new db package.
lending-poc/.gitignore	Ignores local venv/ directory.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 24 changed files in this pull request and generated 1 comment.

Suppressed comments (7)

lending-poc/docker-compose.yml:26

• This second required .env entry also prevents the app service from starting in a clean checkout, despite DATABASE_URL having a Docker default below. Make the file optional so the compose defaults remain usable.

 env_file:
- .env

lending-poc/docker-compose.yml:5

• This makes .env mandatory for the database service. Because .env is gitignored and this PR removes the contents of .env.example, docker compose up from a clean checkout now fails before applying any of the fallback values below. Make the file optional or remove this env_file entry.

This issue also appears on line 25 of the same file.

 env_file:
- .env

lending-poc/app/config.py:12

• The shared .env now needs DB-only keys such as ENCRYPTION_KEY/POSTGRES_*, but app.config.Settings still uses Pydantic's default extra="forbid" for dotenv input. As soon as those new keys are added, importing app.config fails with extra_forbidden; configure this settings class to ignore keys owned by the standalone DB/Compose layer.

 DATABASE_URL: str

lending-poc/db/models/validation_result.py:21

• case_id and document_id are independent foreign keys, so the database accepts a validation result whose document belongs to a different case. This corrupts the case/document relationship represented by the model; enforce the pair with a composite foreign key (and a matching unique constraint on documents) or otherwise validate it transactionally.

 document_id: Mapped[uuid.UUID | None] = mapped_column(
UUID(as_uuid=True), ForeignKey("documents.id", ondelete="CASCADE"), nullable=True, index=True
)

lending-poc/db/config.py:9

• DATABASE_URL is required immediately when this module is imported, while this PR also empties the tracked .env.example. A clean checkout therefore cannot run standalone migrations or import the DB package without an undocumented setting; restore a usable environment template that documents the local/Docker URLs and how to supply a valid Fernet key.

 DATABASE_URL: str
ENCRYPTION_KEY: str = ""

lending-poc/docker-compose.yml:28

• The fallback connection URL is hard-coded to the default credentials/database, so setting any of the newly configurable POSTGRES_USER, POSTGRES_PASSWORD, or POSTGRES_DB values without also duplicating them in DATABASE_URL_DOCKER initializes one database but makes the app connect with different credentials. Derive both services from one coherent connection configuration while handling URL-encoding for credentials.

 DATABASE_URL: ${DATABASE_URL_DOCKER:-postgresql+asyncpg://postgres:postgres@db:5432/lending_poc}

lending-poc/db/models/enums.py:4

• This names app.services.dto as an existing consumer, but there is no app/services package or DTO module in the repository. Keep the ownership guidance generic so the new module documentation does not point contributors to a nonexistent import path.

Owned by the DB layer since they back Postgres enum columns; the pipeline
layer (app.services.dto) imports these rather than redefining them.