diff --git a/apps/dev-playground/config/queries/metric-views.json b/apps/dev-playground/config/metric-views/definitions.json similarity index 70% rename from apps/dev-playground/config/queries/metric-views.json rename to apps/dev-playground/config/metric-views/definitions.json index 75dd7d198..7d8314666 100644 --- a/apps/dev-playground/config/queries/metric-views.json +++ b/apps/dev-playground/config/metric-views/definitions.json @@ -1,4 +1,5 @@ { + "$schema": "https://databricks.github.io/appkit/schemas/metric-source.schema.json", "metricViews": { "revenue": { "source": "appkit_demo.public.revenue_metrics" diff --git a/docs/docs/development/type-generation.md b/docs/docs/development/type-generation.md index b1bdeee74..04b2091c6 100644 --- a/docs/docs/development/type-generation.md +++ b/docs/docs/development/type-generation.md @@ -86,13 +86,13 @@ In blocking mode the generator starts a stopped warehouse, waits (bounded) for i ## Metric-view types -`generate-types` (and the Vite plugin) emit metric-view types **additively** — there is no separate command. When a `config/queries/metric-views.json` file is present, the same run that generates your query types also DESCRIBEs each declared [UC Metric View](../plugins/analytics.md) and writes `metric-views.d.ts` into `shared/appkit-types/`: +`generate-types` (and the Vite plugin) emit metric-view types **additively** — there is no separate command. When a `config/metric-views/definitions.json` file is present, the same run that generates your query types also DESCRIBEs each declared [UC Metric View](../plugins/analytics.md) and writes `metric-views.d.ts` into `shared/appkit-types/`: - `metric-views.d.ts` — augments the `MetricRegistry` interface so `useMetricView('', …)` is autocompleted and type-checked. Each view's measures, dimensions, and their semantic metadata (SQL type, display name, format, time grains) are encoded at the type level. -If `metric-views.json` is absent the metric path stays dormant (nothing is emitted). When present it follows the **same** warehouse-readiness contract as query types: in the default non-blocking run a view that can't be described yet — a cold warehouse, or a bad/unreachable source — is written with permissive types and a warning, while under `--wait` that same situation fails the build so CI never ships incomplete metric types. A malformed `metric-views.json` (invalid JSON, or a source that isn't a three-part UC FQN) fails fast in every mode. +If `config/metric-views/definitions.json` is absent the metric path stays dormant (nothing is emitted). When present it follows the **same** warehouse-readiness contract as query types: in the default non-blocking run a view that can't be described yet — a cold warehouse, or a bad/unreachable source — is written with permissive types and a warning, while under `--wait` that same situation fails the build so CI never ships incomplete metric types. A malformed `definitions.json` (invalid JSON, or a source that isn't a three-part UC FQN) fails fast in every mode. -`metric-views.json` is keyed by metric key; each entry names the three-part UC FQN of the view and, optionally, the executor it runs as (`app_service_principal`, the default, or `user`): +`definitions.json` is keyed by metric key; each entry names the three-part UC FQN of the view and, optionally, the executor it runs as (`app_service_principal`, the default, or `user`): ```json { diff --git a/docs/docs/plugins/analytics.md b/docs/docs/plugins/analytics.md index 0458fedff..a2ca889a2 100644 --- a/docs/docs/plugins/analytics.md +++ b/docs/docs/plugins/analytics.md @@ -113,9 +113,9 @@ The analytics plugin exposes these endpoints (mounted under `/api/analytics`): ## Metric views -`POST /api/analytics/metric/:key` measures a [Unity Catalog Metric View](https://docs.databricks.com/en/metric-views/index.html) that you declared in `config/queries/metric-views.json`. Instead of writing SQL, the caller sends a structured request — which measures to aggregate, which dimensions to group by, and an optional filter — and the plugin builds and runs the `SELECT MEASURE(...) ... GROUP BY ALL` for you against the view. +`POST /api/analytics/metric/:key` measures a [Unity Catalog Metric View](https://docs.databricks.com/en/metric-views/index.html) that you declared in `config/metric-views/definitions.json`. Instead of writing SQL, the caller sends a structured request — which measures to aggregate, which dimensions to group by, and an optional filter — and the plugin builds and runs the `SELECT MEASURE(...) ... GROUP BY ALL` for you against the view. -The route is **dormant until `metric-views.json` exists**: with no config file, every metric key returns `404`. Declaring the file (and generating types) is covered in [Metric-view types](../development/type-generation.md#metric-view-types); this section documents the runtime endpoint that config activates. +The route is **dormant until `config/metric-views/definitions.json` exists**: with no config file, every metric key returns `404`. Declaring the file (and generating types) is covered in [Metric-view types](../development/type-generation.md#metric-view-types); this section documents the runtime endpoint that config activates. ### Request body @@ -133,7 +133,7 @@ Content-Type: application/json } ``` -`:key` is a metric key from `metric-views.json`. The body fields: +`:key` is a metric key from `definitions.json`. The body fields: | Field | Type | Required | Description | | --------------- | ---------- | -------- | ------------------------------------------------------------------------------------------------------------ | @@ -208,7 +208,7 @@ Filters are bounded to keep hostile input from exhausting the server: nesting de ### Executors (cache scope) -Each entry in `metric-views.json` names the executor the query runs as, which also sets the cache scope. This is fixed by config, not the request: +Each entry in `definitions.json` names the executor the query runs as, which also sets the cache scope. This is fixed by config, not the request: | `executor` | Runs as | Cache | | --------------------------------- | ---------------------------------- | ----------------------- | @@ -241,11 +241,11 @@ On failure it emits an `error` event instead. | Status | Body | When | | ------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| `404` | `{ "error": "Metric not found" }` | `:key` is not declared in `metric-views.json` (also the response for every key when the file is absent). | +| `404` | `{ "error": "Metric not found" }` | `:key` is not declared in `definitions.json` (also the response for every key when the file is absent). | | `400` | `{ "error": "Invalid metric request body (fields: …)", "code": … }` | The request body fails validation. The message names only the offending field paths, never the submitted values. | -| `503` | `{ "error": "Metric registry not available", "code": "METRIC_REGISTRY_LOAD_FAILED" }` | `metric-views.json` is present but malformed or unreadable. | +| `503` | `{ "error": "Metric registry not available", "code": "METRIC_REGISTRY_LOAD_FAILED" }` | `definitions.json` is present but malformed or unreadable. | -Editing `metric-views.json` is picked up on the next request — no server restart is needed. A previously malformed file that you fix likewise starts working on the next request. +Editing `definitions.json` is picked up on the next request — no server restart is needed. A previously malformed file that you fix likewise starts working on the next request. ## Frontend usage diff --git a/docs/static/schemas/metric-source.schema.json b/docs/static/schemas/metric-source.schema.json index e606b92da..82f35fbf3 100644 --- a/docs/static/schemas/metric-source.schema.json +++ b/docs/static/schemas/metric-source.schema.json @@ -43,5 +43,5 @@ } }, "additionalProperties": false, - "description": "Schema for AppKit metric-views.json — declares Unity Catalog Metric View sources for the analytics plugin's metric-view path. Each entry under 'metricViews' binds a metric key to a UC metric view FQN and an executor ('app_service_principal' shared cache, or 'user' per-user cache). Object form (rather than bare string) at v1 enables future per-entry option growth without breaking changes." + "description": "Schema for AppKit config/metric-views/definitions.json — declares Unity Catalog Metric View sources for the analytics plugin's metric-view path. Each entry under 'metricViews' binds a metric key to a UC metric view FQN and an executor ('app_service_principal' shared cache, or 'user' per-user cache). Object form (rather than bare string) at v1 enables future per-entry option growth without breaking changes." } diff --git a/packages/appkit/src/app/index.ts b/packages/appkit/src/app/index.ts index ec2260536..19a43927a 100644 --- a/packages/appkit/src/app/index.ts +++ b/packages/appkit/src/app/index.ts @@ -29,17 +29,32 @@ interface FileSystemAdapter { export class AppManager { private readonly _queriesDir: string; + private readonly _metricViewsDir: string; constructor( queriesDir: string = path.resolve(process.cwd(), "config/queries"), + // Metric-view declarations live in a sibling `config/metric-views/` + // directory (next to `config/queries/` and `config/agents/`), NOT inside + // the queries folder. Default it as a sibling of `queriesDir` so a + // single-arg `new AppManager(dir)` (and every test that overrides only the + // queries dir) still resolves the metric-views dir consistently. + metricViewsDir: string = path.resolve( + path.dirname(queriesDir), + "metric-views", + ), ) { this._queriesDir = queriesDir; + this._metricViewsDir = metricViewsDir; } get queriesDir(): string { return this._queriesDir; } + get metricViewsDir(): string { + return this._metricViewsDir; + } + /** * Whether `req` is a dev-tunnel (`?dev`) request. Internal `?dev` predicate * shared by {@link createFsAdapter} (tunnel-vs-`fs` branch) and @@ -50,15 +65,19 @@ export class AppManager { } /** - * Validates that a file path is within the queries directory + * Validates that a file path is within the given base directory. The + * `baseDir` is passed explicitly (rather than pinned to `queriesDir`) so the + * same traversal guard protects reads from any config directory — + * `config/queries/` for `.sql` files and `config/metric-views/` for the + * metric-view definitions. */ - private validatePath(fileName: string): string | null { - const queryFilePath = path.join(this.queriesDir, fileName); - const resolvedPath = path.resolve(queryFilePath); - const resolvedQueriesDir = path.resolve(this.queriesDir); + private validatePath(fileName: string, baseDir: string): string | null { + const filePath = path.join(baseDir, fileName); + const resolvedPath = path.resolve(filePath); + const resolvedBaseDir = path.resolve(baseDir); - if (!resolvedPath.startsWith(resolvedQueriesDir)) { - logger.error("Invalid query path: path traversal detected"); + if (!resolvedPath.startsWith(resolvedBaseDir)) { + logger.error("Invalid config path: path traversal detected"); return null; } @@ -160,7 +179,7 @@ export class AppManager { } // Validate and resolve the file path - const resolvedPath = this.validatePath(queryFileName); + const resolvedPath = this.validatePath(queryFileName, this._queriesDir); if (!resolvedPath) { return null; } @@ -176,20 +195,24 @@ export class AppManager { } /** - * Read a single config file from the queries directory, dev-tunnel-aware. + * Read a single config file from a given base directory, dev-tunnel-aware. + * Shared core behind {@link readConfigFile} (queries dir) and + * {@link readMetricViewsConfig} (metric-views dir). * - * @param fileName - File name (or relative path) within the queries directory. + * @param baseDir - The config directory the file must resolve within. + * @param fileName - File name (or relative path) within `baseDir`. * @param req - Optional request object to detect dev mode. * @param devFileReader - Optional DevFileReader to read via the WebSocket tunnel. * @returns The raw file contents, or `null` when the file is absent / the path is rejected. */ - async readConfigFile( + private async readFileFromDir( + baseDir: string, fileName: string, req?: RequestLike, devFileReader?: DevFileReader, ): Promise { - // Traversal guard: refuse to read outside the queries directory. - const resolvedPath = this.validatePath(fileName); + // Traversal guard: refuse to read outside the base directory. + const resolvedPath = this.validatePath(fileName, baseDir); if (!resolvedPath) { return null; } @@ -208,6 +231,45 @@ export class AppManager { } } + /** + * Read a single config file from the queries directory, dev-tunnel-aware. + * + * @param fileName - File name (or relative path) within the queries directory. + * @param req - Optional request object to detect dev mode. + * @param devFileReader - Optional DevFileReader to read via the WebSocket tunnel. + * @returns The raw file contents, or `null` when the file is absent / the path is rejected. + */ + async readConfigFile( + fileName: string, + req?: RequestLike, + devFileReader?: DevFileReader, + ): Promise { + return this.readFileFromDir(this._queriesDir, fileName, req, devFileReader); + } + + /** + * Read a single config file from the metric-views directory + * (`config/metric-views/`), dev-tunnel-aware. Same absent-file/traversal + * semantics as {@link readConfigFile}, but rooted at {@link metricViewsDir}. + * + * @param fileName - File name (or relative path) within the metric-views directory. + * @param req - Optional request object to detect dev mode. + * @param devFileReader - Optional DevFileReader to read via the WebSocket tunnel. + * @returns The raw file contents, or `null` when the file is absent / the path is rejected. + */ + async readMetricViewsConfig( + fileName: string, + req?: RequestLike, + devFileReader?: DevFileReader, + ): Promise { + return this.readFileFromDir( + this._metricViewsDir, + fileName, + req, + devFileReader, + ); + } + private isNotFoundError(error: unknown, req?: RequestLike): boolean { if ((error as NodeJS.ErrnoException)?.code === "ENOENT") { return true; diff --git a/packages/appkit/src/app/tests/read-config-file.test.ts b/packages/appkit/src/app/tests/read-config-file.test.ts index 0a44e192c..c08ad5a6d 100644 --- a/packages/appkit/src/app/tests/read-config-file.test.ts +++ b/packages/appkit/src/app/tests/read-config-file.test.ts @@ -34,17 +34,38 @@ describe("AppManager.readConfigFile", () => { path.resolve(process.cwd(), "config/queries"), ); }); + + test("metricViewsDir getter exposes the overridden directory", () => { + const mvDir = path.join(tmpDir, "metric-views"); + const manager = new AppManager(tmpDir, mvDir); + expect(manager.metricViewsDir).toBe(mvDir); + }); + + test("metricViewsDir defaults to a sibling of queriesDir", () => { + // Single-arg construction: metric-views sits next to the queries dir. + const manager = new AppManager(tmpDir); + expect(manager.metricViewsDir).toBe( + path.resolve(path.dirname(tmpDir), "metric-views"), + ); + }); + + test("metricViewsDir defaults to /config/metric-views with no override", () => { + const defaultManager = new AppManager(); + expect(defaultManager.metricViewsDir).toBe( + path.resolve(process.cwd(), "config/metric-views"), + ); + }); }); describe("production mode (direct fs)", () => { test("returns file contents for an existing file", async () => { await fs.writeFile( - path.join(tmpDir, "metric-views.json"), + path.join(tmpDir, "sample-config.json"), '{"hello":"world"}', "utf8", ); - const result = await appManager.readConfigFile("metric-views.json"); + const result = await appManager.readConfigFile("sample-config.json"); expect(result).toBe('{"hello":"world"}'); }); @@ -68,13 +89,13 @@ describe("AppManager.readConfigFile", () => { vi.spyOn(fs, "readFile").mockRejectedValueOnce(err); await expect( - appManager.readConfigFile("metric-views.json"), + appManager.readConfigFile("sample-config.json"), ).rejects.toThrow("permission denied"); }); test("reads via direct fs (not the dev reader) without ?dev", async () => { await fs.writeFile( - path.join(tmpDir, "metric-views.json"), + path.join(tmpDir, "sample-config.json"), "prod-contents", "utf8", ); @@ -84,7 +105,7 @@ describe("AppManager.readConfigFile", () => { }; const result = await appManager.readConfigFile( - "metric-views.json", + "sample-config.json", { query: {}, headers: {} }, devFileReader, ); @@ -114,14 +135,14 @@ describe("AppManager.readConfigFile", () => { }; const result = await appManager.readConfigFile( - "metric-views.json", + "sample-config.json", devReq, devFileReader, ); expect(result).toBe("dev-contents"); expect(devFileReader.readFile).toHaveBeenCalledWith( - expect.stringContaining("metric-views.json"), + expect.stringContaining("sample-config.json"), devReq, ); }); @@ -135,7 +156,7 @@ describe("AppManager.readConfigFile", () => { }; const result = await appManager.readConfigFile( - "metric-views.json", + "sample-config.json", devReq, devFileReader, ); @@ -150,8 +171,81 @@ describe("AppManager.readConfigFile", () => { }; await expect( - appManager.readConfigFile("metric-views.json", devReq, devFileReader), + appManager.readConfigFile("sample-config.json", devReq, devFileReader), ).rejects.toThrow("tunnel disconnected"); }); }); + + describe("readMetricViewsConfig (metric-views dir)", () => { + let mvDir: string; + let manager: AppManager; + + beforeEach(async () => { + // Give the metric-views dir its own explicit path so it is independent of + // the queries dir under test above. + mvDir = await fs.mkdtemp(path.join(os.tmpdir(), "appmgr-mv-")); + manager = new AppManager(tmpDir, mvDir); + }); + + afterEach(async () => { + await fs.rm(mvDir, { recursive: true, force: true }); + }); + + test("reads definitions.json from the metric-views dir", async () => { + await fs.writeFile( + path.join(mvDir, "definitions.json"), + '{"metricViews":{}}', + "utf8", + ); + + const result = await manager.readMetricViewsConfig("definitions.json"); + expect(result).toBe('{"metricViews":{}}'); + }); + + test("does NOT read the file from the queries dir", async () => { + // A definitions.json in the queries dir must not resolve — the reader is + // rooted at the metric-views dir only. + await fs.writeFile( + path.join(tmpDir, "definitions.json"), + '{"metricViews":{"stray":{}}}', + "utf8", + ); + + const result = await manager.readMetricViewsConfig("definitions.json"); + expect(result).toBeNull(); + }); + + test("returns null for a genuine not-found (ENOENT)", async () => { + const result = await manager.readMetricViewsConfig("definitions.json"); + expect(result).toBeNull(); + }); + + test("returns null and does not read outside the metric-views dir", async () => { + const readSpy = vi.spyOn(fs, "readFile"); + const result = await manager.readMetricViewsConfig("../../etc/passwd"); + + expect(result).toBeNull(); + expect(readSpy).not.toHaveBeenCalled(); + }); + + test("reads via devFileReader in dev mode", async () => { + const devReq = { query: { dev: "true" }, headers: {} }; + const devFileReader: DevFileReader = { + readdir: vi.fn(), + readFile: vi.fn().mockResolvedValue("dev-mv-contents"), + }; + + const result = await manager.readMetricViewsConfig( + "definitions.json", + devReq, + devFileReader, + ); + + expect(result).toBe("dev-mv-contents"); + expect(devFileReader.readFile).toHaveBeenCalledWith( + expect.stringContaining("definitions.json"), + devReq, + ); + }); + }); }); diff --git a/packages/appkit/src/plugins/analytics/analytics.ts b/packages/appkit/src/plugins/analytics/analytics.ts index 7515fd140..362a74536 100644 --- a/packages/appkit/src/plugins/analytics/analytics.ts +++ b/packages/appkit/src/plugins/analytics/analytics.ts @@ -478,11 +478,11 @@ export class AnalyticsPlugin extends Plugin implements ToolProvider { return; } - // Resolve the registry from disk: read + parse `metric-views.json` once per + // Resolve the registry from disk: read + parse `definitions.json` once per // request (no memoization). Reads through the plugin's shared `this.app` - // (the base `Plugin`'s `AppManager`, rooted at `config/queries/` under the - // process cwd), so this is dev-tunnel aware and inherits the traversal - // guard — same as the sibling `.sql` query path. + // (the base `Plugin`'s `AppManager`) from `config/metric-views/` under the + // process cwd, so this is dev-tunnel aware and inherits the traversal + // guard — the same mechanism as the sibling `.sql` query path. let registry: Record; try { registry = await loadMetricRegistry(this.app, req, this.devFileReader); @@ -535,7 +535,7 @@ export class AnalyticsPlugin extends Plugin implements ToolProvider { } // Lane dispatch. The lane comes from the registration (the entry's - // `executor` in metric-views.json), NOT a URL segment or `.obo.sql` + // `executor` in definitions.json), NOT a URL segment or `.obo.sql` // filename: an OBO-lane metric runs on-behalf-of the requesting user // (per-user cache via `asUser(req)`), an SP-lane metric as the app service // principal (shared cache). diff --git a/packages/appkit/src/plugins/analytics/mv/constants.ts b/packages/appkit/src/plugins/analytics/mv/constants.ts index abd210db7..7e78f1ecd 100644 --- a/packages/appkit/src/plugins/analytics/mv/constants.ts +++ b/packages/appkit/src/plugins/analytics/mv/constants.ts @@ -1,6 +1,6 @@ import type { MetricFilterOperatorName, MetricLane } from "../types"; -export const METRIC_CONFIG_FILE = "metric-views.json"; +export const METRIC_CONFIG_FILE = "definitions.json"; /** * Measure, dimension, and filter-member names are **column identifiers**: they diff --git a/packages/appkit/src/plugins/analytics/mv/registry.ts b/packages/appkit/src/plugins/analytics/mv/registry.ts index 1b44dfc8c..9c236f2ef 100644 --- a/packages/appkit/src/plugins/analytics/mv/registry.ts +++ b/packages/appkit/src/plugins/analytics/mv/registry.ts @@ -1,8 +1,9 @@ import path from "node:path"; // Canonical metric-source schema — the single source of truth for -// `metric-views.json`. Imported from the shared source directly (matching the -// type-generator's runtime, which pulls the zod-free `metric-fqn.ts` from the -// same tree) so the runtime and the generated JSON schema validate identically. +// `config/metric-views/definitions.json`. Imported from the shared source +// directly (matching the type-generator's runtime, which pulls the zod-free +// `metric-fqn.ts` from the same tree) so the runtime and the generated JSON +// schema validate identically. import { metricSourceSchema } from "../../../../../shared/src/schemas/metric-source"; import type { AppManager, DevFileReader, RequestLike } from "../../../app"; import { createLogger } from "../../../logging/logger"; @@ -12,18 +13,18 @@ import { laneFromExecutor, METRIC_CONFIG_FILE } from "./constants"; const logger = createLogger("analytics:metric-views"); /** - * Read and validate `config/queries/metric-views.json` into a metric registry. + * Read and validate `config/metric-views/definitions.json` into a metric registry. * * Async and stateless — registration is a pure config parse with no warehouse * round-trip, no `DESCRIBE`, and no build-time metadata bundle. * - * The file is read **through {@link AppManager.readConfigFile}** rather than - * `node:fs` directly, so this path is dev-tunnel-aware (a `?dev` request reads - * the developer's local file over the WebSocket tunnel) and inherits the + * The file is read **through {@link AppManager.readMetricViewsConfig}** rather + * than `node:fs` directly, so this path is dev-tunnel-aware (a `?dev` request + * reads the developer's local file over the WebSocket tunnel) and inherits the * traversal guard. In production that's a plain `fs.readFile` under the hood, * so the semantics below are unchanged. * - * Absent file -> empty registry (`null` from `readConfigFile`). + * Absent file -> empty registry (`null` from `readMetricViewsConfig`). * Malformed file -> 503 (throws). * * @param app - The {@link AppManager} that resolves + reads the config file. @@ -35,9 +36,13 @@ export async function loadMetricRegistry( req?: RequestLike, devFileReader?: DevFileReader, ): Promise> { - const metricPath = path.join(app.queriesDir, METRIC_CONFIG_FILE); + const metricPath = path.join(app.metricViewsDir, METRIC_CONFIG_FILE); - const raw = await app.readConfigFile(METRIC_CONFIG_FILE, req, devFileReader); + const raw = await app.readMetricViewsConfig( + METRIC_CONFIG_FILE, + req, + devFileReader, + ); if (raw === null) { // Absent file (ENOENT in prod / dev-tunnel not-found) or a rejected // traversal path → dormant. Same as the old ENOENT branch. @@ -49,7 +54,7 @@ export async function loadMetricRegistry( parsed = JSON.parse(raw); } catch (err) { throw new Error( - `Failed to parse metric-views.json at ${metricPath}: ${(err as Error).message}`, + `Failed to parse definitions.json at ${metricPath}: ${(err as Error).message}`, ); } @@ -58,7 +63,7 @@ export async function loadMetricRegistry( const issues = result.error.issues .map((i) => `${i.path.join(".")}: ${i.message}`) .join("; "); - throw new Error(`Invalid metric-views.json at ${metricPath}: ${issues}`); + throw new Error(`Invalid definitions.json at ${metricPath}: ${issues}`); } // Null-prototype map so a metric key that collides with an inherited diff --git a/packages/appkit/src/plugins/analytics/tests/metric.test.ts b/packages/appkit/src/plugins/analytics/tests/metric.test.ts index 41b78079c..2fe78927e 100644 --- a/packages/appkit/src/plugins/analytics/tests/metric.test.ts +++ b/packages/appkit/src/plugins/analytics/tests/metric.test.ts @@ -69,23 +69,25 @@ const tempRegistryDirs: string[] = []; /** * Construct an `AnalyticsPlugin` whose metric-registry gateway is pointed at - * `dir`. + * `dir` (treated as the metric-views config directory). * * The plugin reads the registry through the base `Plugin`'s shared `this.app` - * (an `AppManager` rooted at `config/queries/` under the cwd). There is no - * config field to relocate that directory, so a test points the plugin at a - * fixture dir by overriding the `AppManager` with one constructed over `dir`. - * `app` is `protected` on the base `Plugin`, hence the deliberate test-only - * cast — the single seam every route-handler test threads through. + * (an `AppManager`; the metric path reads from its `metricViewsDir`, + * `config/metric-views/` under the cwd). There is no config field to relocate + * that directory, so a test points the plugin at a fixture dir by overriding + * the `AppManager` with one whose metric-views dir IS `dir` (the first arg — a + * sibling queries dir — is unused by the metric path). `app` is `protected` on + * the base `Plugin`, hence the deliberate test-only cast — the single seam + * every route-handler test threads through. */ function pluginForDir(config: IAnalyticsConfig, dir: string): AnalyticsPlugin { const plugin = new AnalyticsPlugin(config); - (plugin as any).app = new AppManager(dir); + (plugin as any).app = new AppManager(path.join(dir, "queries"), dir); return plugin; } /** - * Write a `metric-views.json` into a fresh temp dir and return the dir, for use + * Write a `definitions.json` into a fresh temp dir and return the dir, for use * with `pluginForDir(config, dir)`. Accepts the internal `MetricRegistration` * shape (matching the old `setRegistry` helper) and maps each entry's `lane` * back to the config's `executor` field. @@ -101,14 +103,14 @@ function registryDir(registry: Record): string { }; } writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews }), ); return dir; } /** - * Overwrite the `metric-views.json` in an existing temp dir (for hot-reload / + * Overwrite the `definitions.json` in an existing temp dir (for hot-reload / * self-heal tests). `raw` lets a test write deliberately malformed content. */ function writeRegistry( @@ -129,7 +131,7 @@ function writeRegistry( ]), ), }); - writeFileSync(path.join(dir, "metric-views.json"), body); + writeFileSync(path.join(dir, "definitions.json"), body); } describe("analytics metric route (Phase 1)", () => { @@ -780,14 +782,14 @@ describe("analytics metric route (Phase 1)", () => { expect(executeMock).toHaveBeenCalled(); }); - test("no metric-views.json present → registry empty, unknown key 404, nothing executes", async () => { + test("no definitions.json present → registry empty, unknown key 404, nothing executes", async () => { const plugin = new AnalyticsPlugin(config); const { router, getHandler } = createMockRouter(); const executeMock = vi.fn(); (plugin as any).SQLClient.executeStatement = executeMock; - // Registry lazily loads from cwd; no config/queries/metric-views.json in - // the test cwd → empty registry (dormant). + // Registry lazily loads from cwd; no config/metric-views/definitions.json + // in the test cwd → empty registry (dormant). plugin.injectRoutes(router); const handler = getHandler("POST", "/metric/:key"); const mockReq = createMockRequest({ @@ -814,21 +816,23 @@ describe("loadMetricRegistry", () => { let app: AppManager; beforeEach(() => { + // `dir` is the metric-views config dir; the loader reads its + // `definitions.json`. The queries dir (first arg) is unused by this path. dir = mkdtempSync(path.join(tmpdir(), "mv-registry-")); - app = new AppManager(dir); + app = new AppManager(path.join(dir, "queries"), dir); }); afterEach(() => { rmSync(dir, { recursive: true, force: true }); }); - test("absent metric-views.json → empty registry (dormancy)", async () => { + test("absent definitions.json → empty registry (dormancy)", async () => { expect(await loadMetricRegistry(app)).toEqual({}); }); test("derives lane from executor (default sp, user → obo)", async () => { writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "cat.sch.revenue_metrics" }, @@ -852,7 +856,7 @@ describe("loadMetricRegistry", () => { test("registry has a null prototype (no inherited-property lookups)", async () => { writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "cat.sch.revenue_metrics" } }, }), @@ -874,19 +878,19 @@ describe("loadMetricRegistry", () => { }); test("malformed JSON throws", async () => { - writeFileSync(path.join(dir, "metric-views.json"), "{ not json"); + writeFileSync(path.join(dir, "definitions.json"), "{ not json"); await expect(loadMetricRegistry(app)).rejects.toThrow(/Failed to parse/); }); test("schema-invalid config throws", async () => { writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "not-a-three-part-fqn" } }, }), ); await expect(loadMetricRegistry(app)).rejects.toThrow( - /Invalid metric-views.json/, + /Invalid definitions.json/, ); }); @@ -899,11 +903,11 @@ describe("loadMetricRegistry", () => { metricViews[`m_${i}`] = { source: `cat.sch.view_${i}` }; } writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews }), ); await expect(loadMetricRegistry(app)).rejects.toThrow( - /Invalid metric-views.json/, + /Invalid definitions.json/, ); }); @@ -913,13 +917,13 @@ describe("loadMetricRegistry", () => { // typegen resolver. const longSegment = "a".repeat(256); writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: `cat.sch.${longSegment}` } }, }), ); await expect(loadMetricRegistry(app)).rejects.toThrow( - /Invalid metric-views.json/, + /Invalid definitions.json/, ); }); @@ -931,7 +935,7 @@ describe("loadMetricRegistry", () => { // One entry with a segment at exactly the 255 limit — must pass. metricViews.m_0 = { source: `cat.sch.${"a".repeat(255)}` }; writeFileSync( - path.join(dir, "metric-views.json"), + path.join(dir, "definitions.json"), JSON.stringify({ metricViews }), ); await expect(loadMetricRegistry(app)).resolves.toBeDefined(); @@ -2055,7 +2059,7 @@ describe("deriveMetricExecutorKey", () => { }); // ── Phase 3: lane dispatch at the handler level. The lane comes from the -// registration (the entry's `executor` in metric-views.json), NOT the URL: +// registration (the entry's `executor` in definitions.json), NOT the URL: // OBO-lane routes through `asUser(req)`, SP-lane through the default executor. // A missing/whitespace OBO identity must land on the canonical 401 envelope, // never an out-of-envelope 500. diff --git a/packages/appkit/src/plugins/analytics/types.ts b/packages/appkit/src/plugins/analytics/types.ts index e91a361f8..83e4f737c 100644 --- a/packages/appkit/src/plugins/analytics/types.ts +++ b/packages/appkit/src/plugins/analytics/types.ts @@ -119,7 +119,7 @@ export interface AnalyticsQueryResponse { /** * Execution lane for a registered metric view, derived from the entry's - * `executor` in `metric-views.json`: + * `executor` in `definitions.json`: * - `"sp"` ← `executor: "app_service_principal"` — queried as the app * service principal (cache shared across all users). * - `"obo"` ← `executor: "user"` — queried on-behalf-of the requesting @@ -128,7 +128,7 @@ export interface AnalyticsQueryResponse { export type MetricLane = "sp" | "obo"; /** - * A single registered metric view, loaded from `config/queries/metric-views.json`. + * A single registered metric view, loaded from `config/metric-views/definitions.json`. * * The registration carries only what the runtime needs to build and dispatch * SQL: the metric `key`, the three-part UC FQN `source`, and the `lane`. There diff --git a/packages/appkit/src/type-generator/index.ts b/packages/appkit/src/type-generator/index.ts index 0a185ae89..2ce7f611d 100644 --- a/packages/appkit/src/type-generator/index.ts +++ b/packages/appkit/src/type-generator/index.ts @@ -269,9 +269,15 @@ async function probeWarehouseState( * only when the warehouse is already RUNNING, otherwise emits permissive * degraded types immediately. `"blocking"` waits for / starts the warehouse * first, failing the build only for a deleted/deleting one. + * @param options.metricViewsFolder - folder that holds `definitions.json` + * (`/config/metric-views`). Optional and independent of `queryFolder`: + * metric-view types generate whenever this folder holds a config, even if the + * app has no `config/queries`. When omitted it defaults to a sibling + * `metric-views` directory of `queryFolder` (so query-only callers keep + * working); when neither is given, the metric path is skipped. * @param options.mvOutFile - optional output file for the MetricRegistry * augmentation. Defaults to a sibling `metric-views.d.ts` file under the same - * directory as `outFile`. Skipped entirely if `metric-views.json` is absent. + * directory as `outFile`. Skipped entirely if `definitions.json` is absent. * @param options.metricFetcher - optional DescribeFetcher used by * {@link syncMetrics} (tests inject a mock; production lazily builds a * default WorkspaceClient-backed one). An injected fetcher always runs: it @@ -281,6 +287,7 @@ async function probeWarehouseState( export async function generateFromEntryPoint(options: { outFile: string; queryFolder?: string; + metricViewsFolder?: string; warehouseId: string; noCache?: boolean; mode?: PreflightMode; @@ -296,6 +303,15 @@ export async function generateFromEntryPoint(options: { mvOutFile, metricFetcher, } = options; + + // Metric config lives in `config/metric-views/`, a sibling of the queries + // folder. Prefer the explicit option; otherwise derive the sibling of + // `queryFolder` so callers that pass only `queryFolder` keep emitting metric + // types. Undefined when neither is given → the metric path stays dormant. + const metricViewsFolder = + options.metricViewsFolder ?? + (queryFolder ? path.resolve(queryFolder, "..", "metric-views") : undefined); + const projectRoot = resolveProjectRoot(outFile); logger.debug("Starting type generation..."); @@ -318,15 +334,18 @@ export async function generateFromEntryPoint(options: { await fs.mkdir(path.dirname(outFile), { recursive: true }); await fs.writeFile(outFile, typeDeclarations, "utf-8"); - // Metric-view types: only emit when metric-views.json exists. - if (queryFolder) { + // Metric-view types: emit whenever a metric-views folder is resolved (gated + // on the metric config's own dir, NOT the queries folder — an app can declare + // metric views without any `.sql` queries). `syncMetricViewsTypes` still + // returns `noConfig` when the folder holds no `definitions.json`. + if (metricViewsFolder) { const mvFile = mvOutFile ?? path.join(path.dirname(outFile), METRIC_TYPES_FILE); let mvResult: SyncMetricViewsTypesResult; try { mvResult = await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId, metricOutFile: mvFile, cache: !noCache, @@ -334,11 +353,11 @@ export async function generateFromEntryPoint(options: { mode, }); } catch (configError) { - // syncMetricViewsTypes only throws for a malformed metric-views.json — re-throw as a message-only TypegenFatalError. + // syncMetricViewsTypes only throws for a malformed definitions.json — re-throw as a message-only TypegenFatalError. throw new TypegenFatalError( [ { - name: "metric-views.json", + name: "config/metric-views/definitions.json", message: getErrorDiagnostic(configError), }, ], @@ -347,7 +366,7 @@ export async function generateFromEntryPoint(options: { } // Deleted/deleting-warehouse fatal preflight (blocking mode only); - // empty (no-op) when metric-views.json is absent or in non-blocking mode. + // empty (no-op) when definitions.json is absent or in non-blocking mode. for (const fe of mvResult.fatalErrors) { fatalErrors.push(fe); } @@ -388,8 +407,8 @@ export interface SyncMetricViewsTypesResult { schemas: MetricSchema[]; failures: MetricSyncFailure[]; /** - * `true` when no `metric-views.json` was found in the query folder, so nothing - * was synced. + * `true` when no `definitions.json` was found in the metric-views folder, so + * nothing was synced. */ noConfig: boolean; /** @@ -409,7 +428,7 @@ export interface SyncMetricViewsTypesResult { * `"describe-now"` mode for a focused, always-converge metric refresh. * * - * @param options.queryFolder - folder that holds `metric-views.json` (`/config/queries`). + * @param options.metricViewsFolder - folder that holds `definitions.json` (`/config/metric-views`). * @param options.warehouseId - SQL warehouse used for `DESCRIBE TABLE EXTENDED`. * @param options.metricOutFile - output path for the MetricRegistry `.d.ts`. * @param options.cache - cache toggle, default ON. Only `cache === false` disables it (so `undefined`/`true` keep caching). @@ -417,7 +436,7 @@ export interface SyncMetricViewsTypesResult { * @param options.mode - preflight/gate policy, default `"describe-now"`. */ export async function syncMetricViewsTypes(options: { - queryFolder: string; + metricViewsFolder: string; warehouseId: string; metricOutFile: string; cache?: boolean; @@ -425,7 +444,7 @@ export async function syncMetricViewsTypes(options: { mode?: "describe-now" | "non-blocking" | "blocking"; }): Promise { const { - queryFolder, + metricViewsFolder, warehouseId, metricOutFile, cache: cacheEnabled, @@ -436,9 +455,9 @@ export async function syncMetricViewsTypes(options: { // Only `cache === false` disables caching; `undefined`/`true` keep it on. const noCache = cacheEnabled === false; - const mvConfig = await readMetricConfig(queryFolder); + const mvConfig = await readMetricConfig(metricViewsFolder); if (!mvConfig) { - // No metric-views.json — additive path stays dormant. The CLI turns this + // No definitions.json — additive path stays dormant. The CLI turns this // into a friendly "nothing to sync" message and exits 0; // generateFromEntryPoint simply ignores `noConfig`. return { schemas: [], failures: [], fatalErrors: [], noConfig: true }; @@ -572,7 +591,7 @@ export async function syncMetricViewsTypes(options: { fetcher, )); - // Surface DESCRIBE failures loudly: a misconfigured metric-views.json would + // Surface DESCRIBE failures loudly: a misconfigured definitions.json would // otherwise silently ship an empty entry that the runtime fail-closed gate // 503s in production. syncMetrics is log-free; this caller is the single // owner of failure logging. diff --git a/packages/appkit/src/type-generator/mv-registry/config.ts b/packages/appkit/src/type-generator/mv-registry/config.ts index 6d4926470..76890bc72 100644 --- a/packages/appkit/src/type-generator/mv-registry/config.ts +++ b/packages/appkit/src/type-generator/mv-registry/config.ts @@ -18,7 +18,7 @@ import type { ResolvedMetricEntry, } from "./types"; -const MV_CONFIG_FILE = "metric-views.json"; +const MV_CONFIG_FILE = "definitions.json"; /** * Safety cap on declared metric views — a typo / DoS guard, NOT a Unity Catalog @@ -44,18 +44,19 @@ function compareKeys(a: string, b: string): number { } /** - * Read {@link MV_CONFIG_FILE} from a queries folder. + * Read {@link MV_CONFIG_FILE} from a metric-views folder + * (`config/metric-views/`). * * Returns `null` if the file does not exist (the metric-view path is - * additive — apps without metric-views.json must not be penalized). There is - * deliberately no fallback to the legacy `metric.json` filename. + * additive — apps without definitions.json must not be penalized). There is + * deliberately no fallback to a legacy filename. * * Throws on JSON parse errors so misconfiguration surfaces loudly. */ export async function readMetricConfig( - queryFolder: string, + metricViewsFolder: string, ): Promise { - const metricPath = path.join(queryFolder, MV_CONFIG_FILE); + const metricPath = path.join(metricViewsFolder, MV_CONFIG_FILE); let raw: string; try { raw = await fs.readFile(metricPath, "utf8"); @@ -71,13 +72,13 @@ export async function readMetricConfig( parsed = JSON.parse(raw); } catch (err) { throw new Error( - `Failed to parse metric-views.json at ${metricPath}: ${(err as Error).message}`, + `Failed to parse definitions.json at ${metricPath}: ${(err as Error).message}`, ); } if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) { throw new Error( - `Invalid metric-views.json at ${metricPath}: expected an object with a 'metricViews' map.`, + `Invalid definitions.json at ${metricPath}: expected an object with a 'metricViews' map.`, ); } @@ -113,7 +114,7 @@ export function resolveMetricConfig( for (const field of Object.keys(config)) { if (!ALLOWED_TOP_LEVEL_FIELDS.has(field)) { throw new Error( - `Invalid top-level field "${field}" in metric-views.json: only '$schema' and 'metricViews' are allowed.`, + `Invalid top-level field "${field}" in definitions.json: only '$schema' and 'metricViews' are allowed.`, ); } } @@ -129,7 +130,7 @@ export function resolveMetricConfig( Array.isArray(metricViews) ) { throw new Error( - `Invalid 'metricViews' in metric-views.json: expected an object map of metric entries.`, + `Invalid 'metricViews' in definitions.json: expected an object map of metric entries.`, ); } @@ -137,7 +138,7 @@ export function resolveMetricConfig( const sortedKeys = Object.keys(metricViews).sort(compareKeys); if (sortedKeys.length > MAX_METRIC_VIEWS) { throw new Error( - `Invalid 'metricViews' in metric-views.json: ${sortedKeys.length} metric views exceed the maximum of ${MAX_METRIC_VIEWS}.`, + `Invalid 'metricViews' in definitions.json: ${sortedKeys.length} metric views exceed the maximum of ${MAX_METRIC_VIEWS}.`, ); } for (const key of sortedKeys) { diff --git a/packages/appkit/src/type-generator/mv-registry/sync.ts b/packages/appkit/src/type-generator/mv-registry/sync.ts index 145f4e519..bc9fd1c29 100644 --- a/packages/appkit/src/type-generator/mv-registry/sync.ts +++ b/packages/appkit/src/type-generator/mv-registry/sync.ts @@ -43,7 +43,7 @@ interface MetricDescribeOutcome { } /** - * Run schema synchronization for every entry in `metric-views.json`. + * Run schema synchronization for every entry in `definitions.json`. */ export async function syncMetrics( resolution: MetricConfigResolution, diff --git a/packages/appkit/src/type-generator/mv-registry/types.ts b/packages/appkit/src/type-generator/mv-registry/types.ts index e47d494b2..fc973c6ff 100644 --- a/packages/appkit/src/type-generator/mv-registry/types.ts +++ b/packages/appkit/src/type-generator/mv-registry/types.ts @@ -11,7 +11,7 @@ import type { DatabricksStatementExecutionResponse } from "../types"; export type MetricLane = "sp" | "obo"; /** - * Single entry in the `metricViews` map of metric-views.json. + * Single entry in the `metricViews` map of definitions.json. * * v1 allows `source` plus the optional `executor`. Object form (rather than * bare string) is the forward-compat seam for future per-entry options @@ -23,7 +23,7 @@ export interface MetricEntryConfig { } /** - * Shape of metric-views.json (mirrors `metricSourceSchema` in + * Shape of definitions.json (mirrors `metricSourceSchema` in * `packages/shared/src/schemas/metric-source.ts`). Inlined here so the * type-generator does not pull in the shared schema package at runtime. */ @@ -82,7 +82,7 @@ export interface MetricColumnMetadata { * time-typed dimensions additionally carry their inferred `timeGrains`. */ export interface MetricSchema { - /** Stable metric key (the map key under `metricViews` in metric-views.json). */ + /** Stable metric key (the map key under `metricViews` in definitions.json). */ key: string; /** Three-part FQN of the metric view. */ source: string; @@ -101,7 +101,7 @@ export interface MetricSchema { degraded?: boolean; } -// Result of reading and resolving metric-views.json — a flat entries list +// Result of reading and resolving definitions.json — a flat entries list // with the lane denormalized for iteration. export interface MetricConfigResolution { entries: ResolvedMetricEntry[]; @@ -121,7 +121,7 @@ export type DescribeFetcher = ( * to the caller so they can decide whether to exit non-zero. */ export interface MetricSyncFailure { - /** Stable metric key — matches the key under `metricViews` in metric-views.json. */ + /** Stable metric key — matches the key under `metricViews` in definitions.json. */ key: string; /** Three-part FQN that failed to resolve. */ source: string; diff --git a/packages/appkit/src/type-generator/tests/index.test.ts b/packages/appkit/src/type-generator/tests/index.test.ts index c469bb115..efcc60f44 100644 --- a/packages/appkit/src/type-generator/tests/index.test.ts +++ b/packages/appkit/src/type-generator/tests/index.test.ts @@ -275,6 +275,10 @@ describe("generateFromEntryPoint — query failure handling", () => { describe("generateFromEntryPoint — metric-view emission", () => { const metricsDir = path.join(__dirname, "__output_metrics__"); const queryFolder = path.join(metricsDir, "queries"); + // The metric config lives in `config/metric-views/` — a sibling of the + // queries folder. `generateFromEntryPoint` derives it from `queryFolder` when + // not passed explicitly, so these tests only pass `queryFolder` below. + const metricViewsFolder = path.join(metricsDir, "metric-views"); const outFile = path.join(metricsDir, "generated", "analytics.d.ts"); // Default: the metric .d.ts is a sibling of `outFile`. const metricFile = path.join(metricsDir, "generated", "metric-views.d.ts"); @@ -302,7 +306,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { const writeMetricConfig = () => { fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "demo.sales.revenue" } }, }), @@ -314,6 +318,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { mocks.cacheFile.contents = undefined; fs.rmSync(metricsDir, { recursive: true, force: true }); fs.mkdirSync(queryFolder, { recursive: true }); + fs.mkdirSync(metricViewsFolder, { recursive: true }); mocks.generateQueriesFromDescribe.mockResolvedValue({ schemas: [], syntaxErrors: [], @@ -325,7 +330,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { fs.rmSync(metricsDir, { recursive: true, force: true }); }); - test("writes metric-views.d.ts when metric-views.json exists", async () => { + test("writes metric-views.d.ts when definitions.json exists", async () => { writeMetricConfig(); await expect( @@ -347,7 +352,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { expect(declarations).toContain('"DECIMAL(38,2)"'); }); - test("emits no metric artifacts and no errors when metric-views.json is absent", async () => { + test("emits no metric artifacts and no errors when definitions.json is absent", async () => { await expect( generateFromEntryPoint({ outFile, @@ -451,7 +456,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { test("non-blocking + warehouse not running: skips all DESCRIBEs but still emits degraded artifacts", async () => { fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "demo.sales.revenue" }, @@ -621,9 +626,9 @@ describe("generateFromEntryPoint — metric-view emission", () => { } }); - test("malformed metric-views.json: a clean TypegenFatalError, not a raw parse error (any mode)", async () => { + test("malformed definitions.json: a clean TypegenFatalError, not a raw parse error (any mode)", async () => { fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), "{ not valid", ); @@ -642,7 +647,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { ); expect(error).toBeInstanceOf(TypegenFatalError); - expect((error as Error).message).toContain("metric-views.json"); + expect((error as Error).message).toContain("definitions.json"); // Query types were written before the metric config was read. expect(fs.existsSync(outFile)).toBe(true); }); @@ -949,7 +954,7 @@ describe("generateFromEntryPoint — metric-view emission", () => { test("empty metricViews map: no probe, no preflight, no client — empty artifacts still ship", async () => { fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews: {} }), ); @@ -1063,6 +1068,9 @@ describe("generateFromEntryPoint — metric-view emission", () => { describe("generateFromEntryPoint — metric cache section", () => { const cacheTestDir = path.join(__dirname, "__output_metric_cache__"); const queryFolder = path.join(cacheTestDir, "queries"); + // Metric config lives in the sibling metric-views folder; generateFromEntryPoint + // derives it from queryFolder when not passed explicitly. + const metricViewsFolder = path.join(cacheTestDir, "metric-views"); const outFile = path.join(cacheTestDir, "generated", "analytics.d.ts"); const metricFile = path.join(cacheTestDir, "generated", "metric-views.d.ts"); @@ -1092,7 +1100,7 @@ describe("generateFromEntryPoint — metric cache section", () => { >, ) => { fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews }), ); }; @@ -1116,6 +1124,7 @@ describe("generateFromEntryPoint — metric cache section", () => { mocks.cacheFile.contents = undefined; fs.rmSync(cacheTestDir, { recursive: true, force: true }); fs.mkdirSync(queryFolder, { recursive: true }); + fs.mkdirSync(metricViewsFolder, { recursive: true }); mocks.generateQueriesFromDescribe.mockResolvedValue({ schemas: [], syntaxErrors: [], diff --git a/packages/appkit/src/type-generator/tests/mv-registry.test.ts b/packages/appkit/src/type-generator/tests/mv-registry.test.ts index 8a06aca68..6fa77693e 100644 --- a/packages/appkit/src/type-generator/tests/mv-registry.test.ts +++ b/packages/appkit/src/type-generator/tests/mv-registry.test.ts @@ -80,7 +80,7 @@ describe("readMetricConfig", () => { await fs.rm(tmpDir, { recursive: true, force: true }); }); - test("returns null when metric-views.json is absent", async () => { + test("returns null when definitions.json is absent", async () => { expect(await readMetricConfig(tmpDir)).toBeNull(); }); @@ -94,9 +94,9 @@ describe("readMetricConfig", () => { expect(await readMetricConfig(tmpDir)).toBeNull(); }); - test("parses a valid metric-views.json", async () => { + test("parses a valid definitions.json", async () => { await fs.writeFile( - path.join(tmpDir, "metric-views.json"), + path.join(tmpDir, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "demo.public.revenue" } }, }), @@ -106,9 +106,9 @@ describe("readMetricConfig", () => { }); test("throws on malformed JSON", async () => { - await fs.writeFile(path.join(tmpDir, "metric-views.json"), "{not json"); + await fs.writeFile(path.join(tmpDir, "definitions.json"), "{not json"); await expect(readMetricConfig(tmpDir)).rejects.toThrowError( - /parse metric-views\.json/, + /parse definitions\.json/, ); }); }); diff --git a/packages/appkit/src/type-generator/tests/sync-metric-views-types.test.ts b/packages/appkit/src/type-generator/tests/sync-metric-views-types.test.ts index dfc650dcc..337892aa8 100644 --- a/packages/appkit/src/type-generator/tests/sync-metric-views-types.test.ts +++ b/packages/appkit/src/type-generator/tests/sync-metric-views-types.test.ts @@ -92,7 +92,7 @@ const DESCRIBE_BY_FQN: Record = { describe("syncMetricViewsTypes", () => { let tmpRoot: string; - let queryFolder: string; + let metricViewsFolder: string; let metricOutFile: string; // A spy fetcher so cache tests can assert which FQNs were (re)described. @@ -106,7 +106,7 @@ describe("syncMetricViewsTypes", () => { const writeMixedConfig = () => { fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews: { // SP lane (default executor). @@ -122,8 +122,8 @@ describe("syncMetricViewsTypes", () => { fetcher.mockClear(); mocks.cacheFile.contents = undefined; tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), "sync-metric-types-")); - queryFolder = path.join(tmpRoot, "config", "queries"); - fs.mkdirSync(queryFolder, { recursive: true }); + metricViewsFolder = path.join(tmpRoot, "config", "metric-views"); + fs.mkdirSync(metricViewsFolder, { recursive: true }); metricOutFile = path.join( tmpRoot, "shared", @@ -140,7 +140,7 @@ describe("syncMetricViewsTypes", () => { writeMixedConfig(); const result = await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -177,9 +177,9 @@ describe("syncMetricViewsTypes", () => { expect(declarations).toContain('"$#,##0.00"'); }); - test("returns noConfig and writes nothing when metric-views.json is absent", async () => { + test("returns noConfig and writes nothing when definitions.json is absent", async () => { const result = await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -198,7 +198,7 @@ describe("syncMetricViewsTypes", () => { // First run: both keys are cache misses → both described, results persisted. await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -210,7 +210,7 @@ describe("syncMetricViewsTypes", () => { // Second run, same config: both keys hit the cache → zero DESCRIBE calls, // and the artifacts are still regenerated from the cached schemas. const result = await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -232,7 +232,7 @@ describe("syncMetricViewsTypes", () => { // Warm the cache. await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -243,7 +243,7 @@ describe("syncMetricViewsTypes", () => { // cache: false ignores the warm section → both keys re-described. await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, cache: false, @@ -257,7 +257,7 @@ describe("syncMetricViewsTypes", () => { // sticky cache entry; the second run must re-describe it rather than ship // the degraded schema. fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "demo.sales.revenue" } }, }), @@ -266,7 +266,7 @@ describe("syncMetricViewsTypes", () => { // First run: fetcher throws → degraded schema + a failure, cached retry:true. fetcher.mockRejectedValueOnce(new Error("TABLE_OR_VIEW_NOT_FOUND")); const first = await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -279,7 +279,7 @@ describe("syncMetricViewsTypes", () => { // Second run, unchanged config, cache ON: the degraded entry is NOT a hit // (degraded !== true clause + retry:true) → re-described, now succeeds. const second = await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -295,7 +295,7 @@ describe("syncMetricViewsTypes", () => { // Warm both keys. await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, @@ -308,14 +308,14 @@ describe("syncMetricViewsTypes", () => { // Shrink the config to a single key. fs.writeFileSync( - path.join(queryFolder, "metric-views.json"), + path.join(metricViewsFolder, "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "demo.sales.revenue" } }, }), ); await syncMetricViewsTypes({ - queryFolder, + metricViewsFolder, warehouseId: "wh-1", metricOutFile, metricFetcher: fetcher, diff --git a/packages/appkit/src/type-generator/tests/vite-plugin.test.ts b/packages/appkit/src/type-generator/tests/vite-plugin.test.ts index 755225561..214dc9a31 100644 --- a/packages/appkit/src/type-generator/tests/vite-plugin.test.ts +++ b/packages/appkit/src/type-generator/tests/vite-plugin.test.ts @@ -262,7 +262,7 @@ describe("appKitTypesPlugin — single-flight generate", () => { expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(2); }); - test("a metric-views.json change triggers a regeneration like a .sql edit", async () => { + test("a definitions.json change in the metric-views folder triggers a regeneration like a .sql edit", async () => { mocks.generateFromEntryPoint.mockResolvedValue(undefined); const plugin = makeConfiguredPlugin(); @@ -278,7 +278,7 @@ describe("appKitTypesPlugin — single-flight generate", () => { // regenerate flow as a .sql edit (no separate machinery). watcher.emit( "change", - path.join(process.cwd(), "config", "queries", "metric-views.json"), + path.join(process.cwd(), "config", "metric-views", "definitions.json"), ); await flush(); expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(2); @@ -296,7 +296,7 @@ describe("appKitTypesPlugin — single-flight generate", () => { await flush(); expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(1); - // Inside the watched folder, but neither .sql nor metric-views.json. + // Inside the watched folder, but neither .sql nor the metric config. watcher.emit( "change", path.join(process.cwd(), "config", "queries", "foo.txt"), @@ -305,7 +305,7 @@ describe("appKitTypesPlugin — single-flight generate", () => { expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(1); }); - test("a legacy-metric-views.json change does NOT regenerate; metric-views.json still does (basename match, not suffix)", async () => { + test("a definitions.json OUTSIDE the metric-views folder does NOT regenerate; one inside does (directory match, not bare basename)", async () => { mocks.generateFromEntryPoint.mockResolvedValue(undefined); const plugin = makeConfiguredPlugin(); @@ -317,19 +317,19 @@ describe("appKitTypesPlugin — single-flight generate", () => { await flush(); expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(1); - // Suffix-matches "metric-views.json" but is a different file — the - // basename check must not fire for it. + // Same basename "definitions.json" but under config/queries/, not the + // metric-views folder — the directory-scoped check must not fire for it. watcher.emit( "change", - path.join(process.cwd(), "config", "queries", "legacy-metric-views.json"), + path.join(process.cwd(), "config", "queries", "definitions.json"), ); await flush(); expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(1); - // The real config file still triggers. + // The real config file (inside config/metric-views/) still triggers. watcher.emit( "change", - path.join(process.cwd(), "config", "queries", "metric-views.json"), + path.join(process.cwd(), "config", "metric-views", "definitions.json"), ); await flush(); expect(mocks.generateFromEntryPoint).toHaveBeenCalledTimes(2); @@ -376,6 +376,10 @@ describe("appKitTypesPlugin — metric option plumbing", () => { process.cwd(), `shared/${TYPES_DIR}/${ANALYTICS_TYPES_FILE}`, ), + // Both config folders are threaded explicitly (not inferred from + // watchFolders ordering): queries + the sibling metric-views dir. + queryFolder: path.join(process.cwd(), "config", "queries"), + metricViewsFolder: path.join(process.cwd(), "config", "metric-views"), mvOutFile: undefined, }), ); diff --git a/packages/appkit/src/type-generator/vite-plugin.ts b/packages/appkit/src/type-generator/vite-plugin.ts index b9d894011..79b239b5e 100644 --- a/packages/appkit/src/type-generator/vite-plugin.ts +++ b/packages/appkit/src/type-generator/vite-plugin.ts @@ -38,7 +38,11 @@ interface AppKitTypesPluginOptions { * Defaults to a sibling of `outFile`, computed by the generator. */ mvOutFile?: string; - /** Folders to watch for changes. */ + /** + * Folders to watch for changes. Defaults to `config/queries` and + * `config/metric-views`. When overridden, include a `queries` folder and/or a + * `metric-views` folder — they are resolved by their trailing path segment. + */ watchFolders?: string[]; } @@ -52,6 +56,11 @@ export function appKitTypesPlugin(options?: AppKitTypesPluginOptions): Plugin { let outFile: string; let mvOutFile: string | undefined; let watchFolders: string[]; + // The queries + metric-views config folders, resolved in `configResolved`. + // Passed explicitly into generateFromEntryPoint so neither is inferred from + // `watchFolders` ordering (which used to assume queries was `watchFolders[0]`). + let queryFolder: string | undefined; + let metricViewsFolder: string | undefined; // Single-flight state for runGenerate(). `inFlight` is the promise of the // currently-running drain (null when idle); `queued` records that a trigger @@ -96,7 +105,8 @@ export function appKitTypesPlugin(options?: AppKitTypesPluginOptions): Plugin { await generateFromEntryPoint({ outFile, - queryFolder: watchFolders[0], + queryFolder, + metricViewsFolder, warehouseId, noCache: false, mode, @@ -290,7 +300,17 @@ export function appKitTypesPlugin(options?: AppKitTypesPluginOptions): Plugin { return false; } - if (!existsSync(path.join(process.cwd(), "config", "queries"))) { + // Run when either config surface exists. Metric-view types are + // independent of `.sql` queries, so a metric-only project (a + // `config/metric-views/` with no `config/queries/`) must still activate + // the plugin. + const hasQueries = existsSync( + path.join(process.cwd(), "config", "queries"), + ); + const hasMetricViews = existsSync( + path.join(process.cwd(), "config", "metric-views"), + ); + if (!hasQueries && !hasMetricViews) { return false; } @@ -313,9 +333,30 @@ export function appKitTypesPlugin(options?: AppKitTypesPluginOptions): Plugin { options?.mvOutFile !== undefined ? path.resolve(projectRoot, options.mvOutFile) : undefined; + + const defaultQueryFolder = path.join(process.cwd(), "config", "queries"); + const defaultMetricViewsFolder = path.join( + process.cwd(), + "config", + "metric-views", + ); watchFolders = options?.watchFolders ?? [ - path.join(process.cwd(), "config", "queries"), + defaultQueryFolder, + defaultMetricViewsFolder, ]; + + // Resolve the two config folders explicitly rather than assuming a + // position in `watchFolders`. With a custom `watchFolders`, match by the + // trailing segment; otherwise use the computed defaults. + if (options?.watchFolders) { + queryFolder = watchFolders.find((f) => path.basename(f) === "queries"); + metricViewsFolder = watchFolders.find( + (f) => path.basename(f) === "metric-views", + ); + } else { + queryFolder = defaultQueryFolder; + metricViewsFolder = defaultMetricViewsFolder; + } }, buildStart() { @@ -343,14 +384,18 @@ export function appKitTypesPlugin(options?: AppKitTypesPluginOptions): Plugin { changedFile.startsWith(folder), ); - if ( - isWatchedFile && - (changedFile.endsWith(".sql") || - // Basename equality, not endsWith: a sibling like - // "legacy-metric-views.json" must not trigger a regenerate — - // only the real config file does. - path.basename(changedFile) === "metric-views.json") - ) { + // The metric config is `definitions.json` — a far more generic name + // than the old `metric-views.json`. Match it by DIRECTORY, not bare + // basename: only a `definitions.json` sitting directly in the + // metric-views folder is the config (a `definitions.json` elsewhere in + // a watched tree must not trigger a regenerate). + const isMetricConfig = + metricViewsFolder !== undefined && + path.basename(changedFile) === "definitions.json" && + path.dirname(path.resolve(changedFile)) === + path.resolve(metricViewsFolder); + + if (isWatchedFile && (changedFile.endsWith(".sql") || isMetricConfig)) { // Route through the single-flight runner (was fire-and-forget // generate(), which could race the initial build / watch). This is a // dev-only hook, so degrade instantly (non-blocking), then re-arm the diff --git a/packages/shared/src/cli/commands/generate-types.test.ts b/packages/shared/src/cli/commands/generate-types.test.ts index 9899b663a..30255cc87 100644 --- a/packages/shared/src/cli/commands/generate-types.test.ts +++ b/packages/shared/src/cli/commands/generate-types.test.ts @@ -108,10 +108,13 @@ describe("generate-types foreground spawn orchestration", () => { vi.clearAllMocks(); acquireSpawnLock.mockReturnValue(true); - // A real temp project root with a config/queries folder so the analytics - // generate path runs. + // A real temp project root with config/queries and config/metric-views + // folders so the analytics + metric generate paths run. tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), "gentypes-")); fs.mkdirSync(path.join(tmpRoot, "config", "queries"), { recursive: true }); + fs.mkdirSync(path.join(tmpRoot, "config", "metric-views"), { + recursive: true, + }); process.env.DATABRICKS_WAREHOUSE_ID = "wh-123"; consoleLog = vi.spyOn(console, "log").mockImplementation(() => {}) as Mock; @@ -231,13 +234,13 @@ describe("generate-types foreground spawn orchestration", () => { expect(acquireSpawnLock).not.toHaveBeenCalled(); }); - test("reports the metric artifact when config/queries/metric-views.json exists", async () => { + test("reports the metric artifact when config/metric-views/definitions.json exists", async () => { // The metric path is additive: generateFromEntryPoint emits metric-views.d.ts // as a sibling of the query out file whenever the config is present. The CLI // announces it off the same dormancy signal. const outFile = path.join(tmpRoot, "shared/appkit-types/analytics.d.ts"); fs.writeFileSync( - path.join(tmpRoot, "config", "queries", "metric-views.json"), + path.join(tmpRoot, "config", "metric-views", "definitions.json"), JSON.stringify({ metricViews: { revenue: { source: "c.s.revenue" } } }), ); @@ -250,7 +253,7 @@ describe("generate-types foreground spawn orchestration", () => { ); }); - test("omits the metric artifact line when metric-views.json is absent (dormant)", async () => { + test("omits the metric artifact line when definitions.json is absent (dormant)", async () => { const outFile = path.join(tmpRoot, "shared/appkit-types/analytics.d.ts"); await runCli([tmpRoot, outFile, "wh-123"]); diff --git a/packages/shared/src/cli/commands/generate-types.ts b/packages/shared/src/cli/commands/generate-types.ts index 004340891..5719caeef 100644 --- a/packages/shared/src/cli/commands/generate-types.ts +++ b/packages/shared/src/cli/commands/generate-types.ts @@ -66,17 +66,31 @@ async function runGenerateTypes( path.join(process.cwd(), "shared/appkit-types/analytics.d.ts"); const queryFolder = path.join(resolvedRootDir, "config/queries"); - if (fs.existsSync(queryFolder)) { + const metricViewsFolder = path.join( + resolvedRootDir, + "config/metric-views", + ); + const hasQueries = fs.existsSync(queryFolder); + const hasMetricViews = fs.existsSync(metricViewsFolder); + + // Generate when either config surface exists. Metric-view types are + // independent of `.sql` queries — an app can declare metric views in + // `config/metric-views/` without a `config/queries/` folder. + if (hasQueries || hasMetricViews) { await typeGen.generateFromEntryPoint({ - queryFolder, + queryFolder: hasQueries ? queryFolder : undefined, + metricViewsFolder: hasMetricViews ? metricViewsFolder : undefined, outFile: resolvedOutFile, warehouseId: resolvedWarehouseId, noCache, mode, }); - console.log(`Generated query types: ${resolvedOutFile}`); - const metricConfig = path.join(queryFolder, "metric-views.json"); + if (hasQueries) { + console.log(`Generated query types: ${resolvedOutFile}`); + } + + const metricConfig = path.join(metricViewsFolder, "definitions.json"); if (fs.existsSync(metricConfig)) { const typesDir = path.dirname(resolvedOutFile); console.log( diff --git a/packages/shared/src/cli/commands/type-generator.d.ts b/packages/shared/src/cli/commands/type-generator.d.ts index 7b3bdd29d..5e7e0a258 100644 --- a/packages/shared/src/cli/commands/type-generator.d.ts +++ b/packages/shared/src/cli/commands/type-generator.d.ts @@ -10,6 +10,7 @@ declare module "@databricks/appkit/type-generator" { export function generateFromEntryPoint(options: { queryFolder?: string; + metricViewsFolder?: string; outFile: string; warehouseId: string; noCache?: boolean; diff --git a/packages/shared/src/schemas/metric-source.ts b/packages/shared/src/schemas/metric-source.ts index 4d64458a7..50595ca0c 100644 --- a/packages/shared/src/schemas/metric-source.ts +++ b/packages/shared/src/schemas/metric-source.ts @@ -1,10 +1,10 @@ /** * AppKit metric-source schema. * - * Single source of truth for `metric-views.json` + * Single source of truth for `config/metric-views/definitions.json` * the config that activates the Analytics' metric-view path. * - * `metric-views.json` declares UC Metric Views under a single `metricViews` map. + * `definitions.json` declares UC Metric Views under a single `metricViews` map. * Each entry binds a metric key to a UC metric view FQN plus the executor * the query runs as: * - `executor: "app_service_principal"` (default) — queried as the app service @@ -112,7 +112,7 @@ export const metricSourceSchema = z }) .strict() .describe( - "Schema for AppKit metric-views.json — declares Unity Catalog Metric View sources for the analytics plugin's metric-view path. Each entry under 'metricViews' binds a metric key to a UC metric view FQN and an executor ('app_service_principal' shared cache, or 'user' per-user cache). Object form (rather than bare string) at v1 enables future per-entry option growth without breaking changes.", + "Schema for AppKit config/metric-views/definitions.json — declares Unity Catalog Metric View sources for the analytics plugin's metric-view path. Each entry under 'metricViews' binds a metric key to a UC metric view FQN and an executor ('app_service_principal' shared cache, or 'user' per-user cache). Object form (rather than bare string) at v1 enables future per-entry option growth without breaking changes.", ) // Caps that cannot be expressed declaratively (zod 4's `z.record` has no // `.max`, and a per-dot-segment length bound isn't a whole-string diff --git a/template/config/metric-views/definitions.json b/template/config/metric-views/definitions.json new file mode 100644 index 000000000..4b8eb357c --- /dev/null +++ b/template/config/metric-views/definitions.json @@ -0,0 +1,6 @@ +{{if .plugins.analytics -}} +{ + "$schema": "https://databricks.github.io/appkit/schemas/metric-source.schema.json", + "metricViews": {} +} +{{- end}}