Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Repository files navigation

Embrasure

“An embrasure is an opening in a fortified wall, built to see and respond without giving up protection.”

CodeQL

Catch unexpected data changes before a dbt PR is reviewed.

Embrasure is open-source, local dbt PR validation for Snowflake, Databricks, and BigQuery:

  • Builds changed models and critical downstream paths in temporary schemas, then cleans them up.
  • Runs dbt tests and compares schema, row counts, null rates, cardinality, ranges, and primary keys with production.
  • Shows affected downstream models and columns.
  • Connects directly to your warehouse, with no Embrasure account or data sent to Embrasure.
  • Includes a verify skill that runs the agent check-and-fix loop.

embrasure auth login uses Snowflake OAuth. Databricks uses a token supplied through the configured environment variable. BigQuery uses Google Application Default Credentials. The warehouse identity needs production read access and permission to create and remove temporary schemas or datasets.

Snowflake quickstart

From your existing Snowflake dbt Core project directory, create or activate its Python environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "dbt-core>=1.5,<2""dbt-snowflake>=1.5,<2""sqlglot>=30,<31"

Then install Embrasure on macOS or Linux with Homebrew:

brew install embrasureai/tap/embrasure
embrasure init
embrasure auth login
embrasure doctor
embrasure check --dry-run
embrasure check

By default, check compares your branch with origin/main.

If dbt is managed by your project

Embrasure uses the dbt Core and Snowflake adapter versions already installed by your project. Activate the project's normal environment, or set dbt.command in embrasure-check.yml to the wrapper your project uses.

Databricks

Install the Databricks dbt adapter instead of dbt-snowflake:

python -m pip install "dbt-core>=1.5,<2""dbt-databricks>=1.5,<2""sqlglot>=30,<31"

Use a version 2 configuration with a typed provider block:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: databrickshost: https://your-workspace.cloud.databricks.comhttp_path: /sql/1.0/warehouses/your-warehouse-idcatalog: analyticsproduction_schema: prodauth:
type: tokentoken_env: DATABRICKS_TOKEN

The integration uses a Databricks SQL warehouse and Unity Catalog. Set DATABRICKS_TOKEN, then run embrasure doctor and embrasure check. Incremental baselines currently require managed Delta tables and use Unity Catalog shallow clones; doctor verifies that a suitable table and grants are available.

BigQuery

Install the BigQuery dbt adapter:

python -m pip install "dbt-core>=1.5,<2""dbt-bigquery>=1.5,<2""sqlglot>=30,<31"

For local development, create Google Application Default Credentials, then initialize Embrasure from the dbt project root:

gcloud auth application-default login
embrasure init
embrasure doctor
embrasure check --dry-run
embrasure check

init detects an active BigQuery dbt profile and writes the version 2 provider configuration. The equivalent manual configuration is:

version: 2dbt:
project_dir: .profile: analyticsaccounts:
- name: primaryprovider:
type: bigqueryproject: analytics-prodlocation: USproduction_schema: prodmaximum_bytes_billed: 10737418240auth:
type: application_default

Application Default Credentials also support GOOGLE_APPLICATION_CREDENTIALS and an attached Google Cloud service account. maximum_bytes_billed applies the same per-query cap to dbt builds and Embrasure comparison queries. Incremental baselines use BigQuery table clones; clone mode seeds candidates with table copies. The source and temporary datasets must be in the same location, and table-clone restrictions still apply.

If you do not use Homebrew, use the installer:

curl -fsSL https://raw.githubusercontent.com/EmbrasureAI/embrasure-cli/main/install.sh | sh

The installer writes to /usr/local/bin when writable, otherwise ~/.local/bin. Set EMBRASURE_INSTALL_DIR to choose another directory.

Installing on Windows (Windows 11 or Windows Server 2022+)

Download the PowerShell installer from GitHub Releases, inspect it, then run it:

$installer=Join-Path$env:TEMP'embrasure-install.ps1'Invoke-WebRequest https://github.com/EmbrasureAI/embrasure-cli/releases/latest/download/install.ps1 -OutFile $installerGet-Content$installerUnblock-File$installer&$installer

The installer verifies the release checksum, installs Embrasure under %LOCALAPPDATA%\Programs\Embrasure, and adds its bin directory to your user PATH. It does not need administrator access. Open a new terminal when it finishes.

Unblock-File removes the internet-zone marker after you inspect the script. It does not change PowerShell's execution policy or verify the publisher. If your organization blocks the script, use the portable ZIP from the same release.

Installer options:

  • Pin a release: & $installer -Version 0.5.4
  • Run without prompts: & $installer -Quiet
  • Uninstall: & $installer -Uninstall

Uninstalling removes Embrasure and its PATH entry, but keeps your configuration, credentials, reports, and logs.

Scoop users can install from Embrasure's official bucket:

scoop bucket add embrasure https://github.com/EmbrasureAI/scoop-bucket
scoop install embrasure/embrasure

WinGet manifests are generated with each release. After the first WinGet listing is accepted:

winget install --id EmbrasureAI.Embrasure --exact

Use Embrasure from an existing Snowflake, Databricks, or BigQuery dbt project whose unchanged production models are already materialized. Embrasure uses those existing relations as the comparison baseline.

For Snowflake and BigQuery, init reads the active dbt profile and asks only for missing values. Databricks uses the version 2 configuration shown above. Use --config <path> before or after any subcommand to choose another config file.

Continue only when embrasure doctor reports READY. Embrasure generates a temporary dbt profile for its own runs; it does not modify your existing profile or production models.

If dbt is installed in .venv, run source .venv/bin/activate in each new shell before using Embrasure.

Example result:

✓ Safe to continue
The change passed across every selected dbt model.
Scope
5 affected models
2 selected for validation
Evidence
2 / 2 models built · 2 compared with production
151,615,312 candidate rows evaluated
Schema, row counts, nulls, cardinality, and distributions checked
1 primary key checked
0 findings · 3 unvalidated models
Temporary warehouse schema removed
Lineage impact
fct_orders
└─ finance_daily
├─ executive_revenue
├─ regional_margin
└─ revenue_forecast_input
Completed in 2m11s

In an interactive terminal, check shows live progress while it runs. Redirected output, CI, and --json stay plain.

Focus and preview

The default validates changed models and every path to a critical model. Critical targets are tagged critical, configured with critical: true, or used directly by a dbt exposure. Use --downstream all for every downstream model or --downstream none for changed models only. Impact is always computed from the full changed set.

Intersect the changed set with one or more explicit models:

embrasure check --select orders --select order_items
embrasure check --select orders --downstream none

An unknown, ambiguous, unchanged, or out-of-scope selection fails instead of returning a misleading pass.

Preview the plan without creating schemas or querying warehouse data:

embrasure check --dry-run
embrasure check --dry-run --json

Dry runs use local dbt parsing but do not resolve credentials, create warehouse schemas, or query warehouse data.

Reports and exit codes

JSON output is versioned and stably ordered. Progress goes to stderr, so stdout contains one JSON document.

embrasure check --json
embrasure check --json --markdown embrasure-check.md
embrasure check --json --report-version 1

Published contracts: v1, v2, v3, and v4. V4 adds column lineage and bounded warehouse execution links and is the default; older versions remain available with --report-version.

Exit codeMeaning
0The check passed
1The data or code needs a fix
2A requested check could not be completed
3Setup, execution, or cleanup failed

Agent loop:

Run `embrasure check --base origin/main --json`.
Exit 1: fix every finding and rerun.
Exit 2: resolve or explain every coverage gap.
Exit 3: fix the setup or execution failure.
Request review only after exit 0.

For a faster first pass on large tables, add --mode quick. Quick mode estimates cardinality and skips percentiles. Deep mode is the default. Primary-key integrity stays exact in both modes.

Arbitrary SQL checks

Query-diff checks compare any two read-only query results exactly. production_sql defaults to sql, and each dbt ref() is rendered against the candidate or production-state manifest. Checks with no refs, or definitions changed since --base, run even when no model changed.

checks:
- type: query_diffname: paid order totalssql: | select customer_id, sum(amount) as paid_amount from {{ ref('orders') }} where status = 'paid' group by customer_idprimary_key: [customer_id]

With a primary key, Embrasure reports added, removed, and changed rows plus per-column mismatch counts. Null or duplicate keys block the value join. Without a key, grouped rows and their multiplicities preserve duplicate-only differences. Query examples are bounded by the configured sample, column, and value limits.

Only persisted dbt models are supported in ref(). Query checks accept one SELECT, WITH, or VALUES expression; other Jinja and multi-statement SQL are rejected. Removed checks are reported as incomplete coverage instead of silently passing.

GitHub Actions

The composite action installs Embrasure. Install dbt with your project's normal locked setup; this example uses requirements.txt. Keep the check in a visible run step so exit codes and secrets remain explicit.

In embrasure-check.yml, configure the account to read the CI secret:

auth:
type: programmatic_access_tokentoken_env: SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN
jobs:
embrasure:
runs-on: ubuntu-24.04permissions:
contents: readsteps:
- uses: actions/checkout@v4with:
fetch-depth: 0
- uses: actions/setup-python@v6with:
python-version: "3.12"
- run: python3 -m pip install -r requirements.txt
- uses: EmbrasureAI/embrasure-cli@v1
- run: embrasure check --base origin/main --jsonenv:
SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN: ${{ secrets.SNOWFLAKE_PROGRAMMATIC_ACCESS_TOKEN }}

fetch-depth: 0 is required because selection compares the working tree with the base revision.

Maintenance

List managed temporary schemas older than six hours:

embrasure clean
embrasure clean --older-than 24 --yes

clean searches only configured account databases and verifies the prefix and ownership marker before removal.

Check for or install an update:

embrasure update --check
embrasure update

Generate shell completion scripts:

embrasure completion bash
embrasure completion zsh
embrasure completion fish
embrasure completion powershell

Troubleshooting

Generated schema is outside the run namespace

Your generate_schema_name macro must preserve the complete target schema. Make custom schemas children of target.schema; do not replace it.

Incremental relation cannot be cloned

Every existing incremental model needs a stable baseline copy, including in full-refresh mode. Snowflake supports tables that can be zero-copy cloned; Databricks currently supports managed Delta tables that can be shallow cloned; BigQuery supports base tables, table clones, and table snapshots accepted by CREATE TABLE CLONE. Use another materialization or exclude an unsupported relation from this validation path.

Incremental candidate seeding fails

The validation role needs SELECT on the production source and CREATE TABLE in the target schema. Run embrasure doctor. If the relation should not use clone mode, rerun with --incremental-mode full-refresh.

Configuration and safety

Use --config <path> before or after any subcommand to choose another config file.

See the example configuration and enterprise setup guide for service credentials, multiple accounts, model policies, filters, thresholds, concurrency, external changes, cross-account dependencies, Metabase, and grants.

Every temporary schema has a unique name and ownership marker. Query results are materialized in a dedicated run-owned schema so they cannot collide with dbt model aliases. Embrasure checks ownership before removal and treats cleanup failures as execution failures. Use a dedicated identity that can read and clone only the production tables under test and create temporary schemas in the required databases or catalogs. SQL validation is not a side-effect sandbox, so the identity must not be able to call unsafe procedures, functions, or external integrations.

Security and data flow documents network connections, local files, returned data, credentials, cleanup, updates, and release verification.

Current limits

  • Databricks support requires a Unity Catalog SQL warehouse and environment-supplied token authentication.
  • Native Windows support requires 64-bit Windows 11 or Windows Server 2022+. Windows 10, Windows on Arm, and machine-wide installation are not supported.
  • Column lineage covers compiled dbt SQL that SQLGlot can resolve. Wildcards without an input schema and dynamic SQL are reported as unresolved.
  • Dashboard column lineage is not inferred from model lineage.
  • Metabase matching covers native SQL cards that reference fully qualified production relations. Unsupported or inaccessible metadata becomes a coverage gap.

Development

Rust 1.88 or newer is required when building from source.

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --locked

The opt-in Snowflake suite uses EMBRASURE_RUN_SNOWFLAKE_TESTS=1 and the EMBRASURE_TEST_SNOWFLAKE_* account, user, role, database, warehouse, and token variables. It covers exact keyed passes and changes, duplicate-only unkeyed differences, incremental cloning, cleanup, and 100,000 synthetic rows.

Contributions are welcome under the Apache 2.0 license.

About

Embrasure: validate dbt changes against real Snowflake data from the command line.

Topics

Resources

Contributing

Security policy

Stars

16 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages