Uh oh!
There was an error while loading. Please reload this page.
feat(sea): type async-execute napi surface + pecotesting e2e - #394
feat(sea): type async-execute napi surface + pecotesting e2e#394msrathore-db wants to merge 3 commits into
Conversation
1056dfa to
4e673fdCompare2538c19 to
e314073Compare4e673fd to
7d07de3Comparee314073 to
e21a4bbCompare7d07de3 to
8c5618cComparee21a4bb to
ca04a30CompareMirrors the new napi `submitStatement` / `AsyncStatement` / `AsyncResultHandle` shapes from kernel branch `msrathore/krn-async-execute` (commit `1957878`) on the JS side and adds an end-to-end test against pecotesting. Plumbing: - `SeaNativeLoader.ts` — new typed interfaces `SeaNativeStatementStatus` (string union mirroring the kernel `StatementStatus` variant names), `SeaNativeAsyncStatement`, `SeaNativeAsyncResultHandle`. `SeaNativeConnection` gains the `submitStatement(sql, options?)` method. - `native/sea/index.d.ts` — adds `AsyncStatement` / `AsyncResultHandle` class declarations and the `Connection.submitStatement(...)` method. Mirrors the napi-rs codegen exactly. - `tests/e2e/sea/async-execute-e2e.test.ts` — three cases: 1. submit → awaitResult → drain 100 rows; assert byte-decoded Arrow row count matches the range(0, 100) query. 2. status() returns a string variant from the kernel enum (Pending/Running/Succeeded/Closed depending on warehouse responsiveness). 3. cancel() against a still-pending statement (range(0, 100_000_000)) completes within a 2s wire-latency budget. Deferred to a follow-on PR: a full `SeaAsyncOperationBackend` implementing the facade's `IOperationBackend` interface so `DBSQLOperation` async-mode (matching Thrift's `IDBSQLOperation` polling-mode surface) flows transparently. The kernel + napi surface is ready; the JS adapter is the next increment. Cross-PR dependency: requires kernel branch `msrathore/krn-async-execute` (commit `1957878`). Stacks on `msrathore/sea-statement-options` (F3 + F4). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Same skip-gate fix as F2/F4 (DA round 1 H1): - `before()` verifies `native/sea/index.linux-x64-gnu.node` exists, skips with a clear "run yarn build:native" message if absent. - `loadBinding()` uses `createRequire(import.meta.url)` so the require works under both CJS and the ESM-reparse path mocha 11+ may use (MODULE_TYPELESS_PACKAGE_JSON reparse-as-ESM). Pre-emptive fix even though F3b wasn't on the DA fixup batch list — the same defect pattern would have been flagged on the next round. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Same ESM-reparse fix as F2/F4. Verified live e2e passes (submit → awaitResult → drain, status() variant, cancel-while-pending). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
ca04a30 to
de55b96Comparemsrathore-db
commented
May 26, 2026
Re-review requested — F1-F4 stack underneath this PR is now DA-approved at round-3; this PR (F3b NodeJS async-execute) is the next adversarial-review target. Round-3 final tip: This PR was parked behind the F1-F4 quality bar per team-lead's hard-stop directive. With F1-F4 + matrix now in APPROVED FOR MERGE state, F3b adversarial review starts on this PR + the kernel companion (#76). Findings will be addressed per the same protocol that closed F1-F4 round-1 → round-2 → round-3:
DA preview notes on F3b (Q1 IPC encoder lift to Co-authored-by: Isaac |
msrathore-db
commented
May 31, 2026
Superseded by #406, which implements the async submit+poll lifecycle (Thrift |
Summary
JS-side typing + live e2e test for the new napi async-execute surface delivered by kernel branch
msrathore/krn-async-execute(commit1957878, PR https://github.com/databricks/databricks-sql-kernel/pull/76).Plumbing
SeaNativeLoader.ts— new typed interfaces:SeaNativeStatementStatus— string union mirroring the kernelStatementStatusvariant names ('Pending' | 'Running' | 'Succeeded' | 'Failed' | 'Cancelled' | 'Closed' | 'Unknown')SeaNativeAsyncStatement—statementId,status(),awaitResult(),cancel(),close()SeaNativeAsyncResultHandle—statementId,fetchNextBatch(),schema()(no cancel/close — parent owns lifecycle)SeaNativeConnection.submitStatement(sql, options?)methodnative/sea/index.d.ts— mirrors the napi-rs codegen: newAsyncStatement/AsyncResultHandleclass declarations +Connection.submitStatement(...)method.tests/e2e/sea/async-execute-e2e.test.ts— three live-warehouse cases:range(0, 100); byte-decode the Arrow IPC and assert row countrange(0, 100_000_000)) completes within 2s wire-latency budgetDeferred (follow-on PR)
A full
SeaAsyncOperationBackendimplementingIOperationBackendsoDBSQLOperationasync-mode flows through the same facade callers use for Thrift. The kernel + napi surface is ready; the facade-level integration is the next increment. This PR ships the kernel→napi→JS shape and locks the behaviour with a live e2e.ULTRATHINK on async-cancel safety
Kernel's
AwaitResultCancelGuard(src/statement/executed_async.rs:305-356) installs an RAII drop hook inawait_result()that fires a fire-and-forgetcancel_statementif the future is dropped mid-poll. This covers:Promise.racelosersAbortController.abort()-driven cancellationPromise.race([awaitResult(), timer]))The napi
util::guardedcatch_unwindcovers V8 panic-across-boundary on top. TheArc<Mutex<Option<…>>>lock shape in the napiAsyncStatementwrapper matches the existingConnectionandStatementso cancel/close queue behind any in-flightawaitResult()until it returns naturally — the kernel handles the drop-the-future case independently.Cross-repo dependency
Requires kernel branch
msrathore/krn-async-execute(PR https://github.com/databricks/databricks-sql-kernel/pull/76). Stacks onmsrathore/sea-statement-options(F3 + F4, PR #393).Test plan