Uh oh!
There was an error while loading. Please reload this page.
chore: extract the Sqlite ODBC driver from the stackable-odbc-rs workspace - #1
Merged
Conversation
Relocates the driver from the stackable-odbc-rs workspace, where it was developed alongside the core framework, into its own repository. The sources are moved verbatim; the manifest is rewritten to stand alone, replacing workspace inheritance with explicit values and adding the package metadata the workspace crate never carried. stackable-odbc-core is a path dependency to the sibling checkout: it is private and unpublished. This commit does not compile. The core API has moved ahead of what the workspace pinned, and adapting to it is the next commit, kept separate so that this one is reviewable as the pure relocation it is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
default_get_info and common_get_info_raw are now generic over the backend, and the SQLEndTran cursor behaviours are derived from two new Backend hooks rather than hard-coded. SqliteBackend overrides both hooks explicitly rather than inheriting the Preserve default, because here the value is observable: this driver reports SQL_TC_DML and implements real transactions, where the Trino driver reports SQL_TC_NONE and never reaches the question. Preserve is correct for a reason specific to this driver rather than to SQLite. exec_direct materialises result sets eagerly, so no rusqlite::Statement is live when end_tran runs: COMMIT cannot fail with SQLITE_BUSY on a pending write, and ROLLBACK cannot abort a pending read with SQLITE_ABORT. Raw SQLite would make rollback SQL_CB_CLOSE. A test pins both values through the FFI entry point so a move to lazy streaming fails loudly. This changes what applications observe. SQL_CURSOR_COMMIT_BEHAVIOR previously reported SQL_CB_DELETE, which was never true -- core advertised it and implemented nothing, so the driver claimed to destroy cursors on commit while preserving them. Three tests called SQLCloseCursor after an INSERT and asserted success. Core now returns 24000 there, correctly: a statement producing no result set never opens a cursor, so there is nothing to close and the handle is already reusable. The calls were never necessary and are removed; one new test covers the corrected behaviour directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Carries over the workspace's clippy, markdownlint, pre-commit and cargo-deny configuration, de-workspaced: cargo test replaces cargo test --workspace, and the packaging output path loses its crates/ prefix. Drops the RUSTSEC-2024-0436 advisory ignore. It covered the unmaintained paste crate reaching the workspace through trino-rust-client, which is not in this driver's dependency tree. Adds a cargo-sort hook, matching the sibling driver repository. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The install and uninstall scripts move unchanged. build-archives.sh loses its workspace assumptions: the crate-relative and repository-root paths collapse into one derived from the script's own location, the dist directory drops its crates/ prefix, and the cargo invocations in the error messages drop -p. Version strings in the packaging README are set to 0.0.1 in the forms release.toml's pre-release-replacements expect, and the support link points at this repository. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite moves from test/sqlite/ to test/, with the directory prefix and the -p crate selectors dropped from the setup and run scripts and from the usage docstrings. PROJECT_DIR is derived from one parent directory rather than two: the scripts sat two levels below the workspace root and now sit one below the repository root. Only the tracked files move. odbc.ini, odbcinst.ini and test.db are generated by setup.sh and stay ignored; the checked-in copies held absolute paths into the old workspace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The libvirt VM definition, its Ansible playbook and the unattended-install configuration move unchanged -- none of it is backend-specific. WINDOWS.md documented both drivers and is rewritten to cover only SQLite: the Trino invocations, the four-config Trino matrix, the Trino connection parameters and the Trino driver registration and DSN examples are gone, and the test path drops its test/sqlite/ prefix. openssl_legacy.cnf moves too. It enables OpenSSL's legacy provider for the MD4 that WinRM's NTLM authentication needs, so it is required by this driver's own windows_test.py rather than being TLS configuration for any particular backend. Corrects PROJECT_DIR in test/windows_test.py, which resolved two parents up from the script and now resolves one: the script sat two levels below the workspace root and now sits one below the repository root. It located windows/openssl_legacy.cnf through that path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Carried over from the workspace and de-workspaced: cargo test replaces cargo test --workspace, the -p selectors and crates/ prefixes are gone, and the release tag trigger drops the sqlite- prefix that disambiguated two drivers releasing from one repository. The SQLite integration suite runs in CI. The sibling Trino repository dropped its equivalent because a Dockerised Trino and Postgres exceed a standard runner; this suite needs only unixODBC and the sqlite3 CLI. The Miri job is not carried over. It only ever ran against stackable-odbc-core -- the driver crates link C libraries Miri cannot execute -- and belongs to that repository. These workflows fail until stackable-odbc-core is reachable to the runner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The README is rewritten to stand alone: it no longer links a sibling crate by relative path, no longer positions the driver against another driver, and points at this repository for support. AGENTS.md is rewritten for this crate alone. The workspace guide covered all three crates; adding an ODBC function, adding a driver, Miri and fuzzing all belong to stackable-odbc-core. New sections record why both cursor-behaviour hooks return Preserve, that eager materialisation is load-bearing rather than an implementation detail, and that this crate deliberately takes no direct odbc-sys dependency. CHANGELOG.md is new, following Keep a Changelog, and records the extraction along with the corrected cursor-behaviour and SQLCloseCursor values. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bumps the version, rewrites CHANGELOG.md and packaging/README.md, then commits, tags and pushes. Publication to crates.io is disabled: the tag triggers the release workflow, which builds both binaries and publishes a GitHub Release with the archives. Tags and commits are signed by configuration rather than by the releaser's git settings, so an unsigned tag fails loudly instead of being produced silently, and allow-branch keeps a stray --execute from tagging whichever branch happens to be checked out. Corrects the release.sh header comment, which named README.md as a file a release rewrites; release.toml rewrites packaging/README.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Doc comments in escape_dialect.rs and ffi_integration_tests.rs pointed at crates/stackable-odbc-sqlite/src/..., and the benchmark usage lines still carried -p crate selectors. A CI comment compared this repository's integration job to the sibling driver's; it now states the property that actually matters, which is that the suite needs no server or container. CHANGELOG.md still names the stackable-odbc-rs workspace. That is deliberate: it records where the code came from, which is what a changelog entry for an extraction is for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Audits all 64 values this driver reports for SQLGetInfo -- including those answered by stackable-odbc-core's defaults, which an application cannot distinguish from the driver's own -- against the SQLGetInfo specification table and SQLite's documented behaviour. SQL_CATALOG_TERM, SQL_CATALOG_NAME_SEPARATOR and SQL_SCHEMA_TERM named a catalog, a separator and a schema while SQL_CATALOG_NAME, SQL_CATALOG_LOCATION, SQL_CATALOG_USAGE and SQL_SCHEMA_USAGE all declared that neither exists. The spec is explicit that the first three are empty strings when the data source supports neither. The driver answered four of the seven itself and let the other three fall through to core's defaults, so nothing held them together. All seven now derive from a single SUPPORTS_CATALOGS / SUPPORTS_SCHEMAS pair rather than being restated, and a test asserts the spec's rule rather than the current values, so it keeps holding if either constant flips. SQL_OUTER_JOIN_CAPABILITIES reported 0 -- no outer joins at all -- while this driver's own SQL_OUTER_JOINS reported "Y". It now reports the seven SQL_OJ_* bits SQLite implements, each proved by executing the join it describes against the bundled library rather than assumed from release notes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
StatementBackend is not in scope in backend.rs, so the link in cursor_commit_behavior's docs did not resolve and cargo doc warned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SQL_ALTER_TABLE inherited stackable-odbc-core's default of 0, which said SQLite cannot alter a table in any way. It now reports ADD COLUMN with DEFAULT and COLLATE, ADD CONSTRAINT, and the CONSTRAINT name definition that clause carries. Every claimed bit is proved by executing the clause against the bundled library, and every unclaimed bit by that library rejecting it. The negative half earned its keep immediately: ADD CONSTRAINT and DROP CONSTRAINT were written off as absent from SQLite's grammar, which is true of 3.51.3 and false of the bundled 3.53.2. A bitmap that only checks what it claims would have understated indefinitely. ADD CONSTRAINT is additionally checked against the stored schema, not just for acceptance: SQLite's ADD clause makes the COLUMN keyword optional, so a statement adding a column called CONSTRAINT would otherwise satisfy the probe. Unqualified DROP COLUMN and DROP CONSTRAINT stay out of the bitmap. SQLite supports both, but the ODBC value offers only CASCADE and RESTRICT variants and SQLite rejects both keywords, so claiming either would advertise syntax an application would send and have refused. RENAME TO and RENAME COLUMN have no bit at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SQL_TXN_ISOLATION_OPTION claimed READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ and SERIALIZABLE. SQLite provides the last of these: "Transactions in SQLite are SERIALIZABLE". READ COMMITTED and REPEATABLE READ are not SQLite concepts at all, and READ UNCOMMITTED additionally requires shared-cache mode -- "the only way that one database connection can see uncommitted changes on a different database connection" -- which SqliteBackend::connect never enables, opening with a plain rusqlite::Connection::open. The overclaim mattered because nothing applies what an application asks for. SQL_ATTR_TXN_ISOLATION is stored on the connection and read back unchanged, never pushed to SQLite, so an application that set REPEATABLE READ was told it had it while running serializable. A test now asserts that SQL_DEFAULT_TXN_ISOLATION names a level SQL_TXN_ISOLATION_OPTION actually offers, which is the invariant the two values were breaking. Spec: https://www.sqlite.org/isolation.html Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
stackable-odbc-core made fourteen Backend methods required, on the grounds that a defaulted capability is a claim no backend ever made. This driver now answers all of them. Six carry values this repository had already established, moved from sqlite_get_info arms and local constants into the hooks: supports_catalogs, supports_schemas, alter_table_support, outer_join_capabilities, default_txn_isolation and txn_isolation_options. The seven catalog and schema info types are no longer answered here at all -- core derives the whole group from the two booleans, and the snapshot is unchanged, which is the evidence that the derivation reproduces what the arms produced. Eight are new. group_by, null_collation, correlation_name, non_nullable_columns and expressions_in_order_by were each verified against the bundled library rather than reasoned about; the two timedate interval bitmaps report 0, matching SQL_TIMEDATE_FUNCTIONS, which claims neither TIMESTAMPADD nor TIMESTAMPDIFF. sql_conformance drops SQL_SC_SQL92_ENTRY for 0. That value was core's default, not an assessment of SQLite, and the spec ties entry level to SQL_GB_GROUP_BY_EQUALS_SELECT -- while SQLite accepts a bare non-aggregated column absent from GROUP BY, and a GROUP BY column absent from the select list. Both are asserted. Claiming no level is the honest answer; raising it means auditing entry-level conformance rather than restoring an invented value. SQL_ALTER_TABLE gains SQL_AT_ADD_CONSTRAINT, which core did not define when this bitmap was first written. The bit means ADD COLUMN with column constraints, not table constraints, and SQLite takes NOT NULL, CHECK, REFERENCES and a named CONSTRAINT there. The four deferrability bits stay unclaimed: SQLite implements deferred constraints only inside a foreign-key clause, and its parser accepts DEFERRABLE after CHECK and NOT NULL where it has no effect -- accepting a token is not implementing the attribute. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SQL_INTEGRITY reported "N", inherited from stackable-odbc-core's default. SQLite implements the entire Integrity Enhancement Facility -- PRIMARY KEY, UNIQUE, NOT NULL, CHECK, DEFAULT, and FOREIGN KEY with referential actions -- and this build enforces every part of it. The earlier justification for "N" was that SQLite leaves foreign keys off unless an application turns them on. That is true of plain SQLite and false here: the bundled library is compiled with SQLITE_DEFAULT_FOREIGN_KEYS, so PRAGMA foreign_keys is already on when a connection opens, and an orphan insert fails without the driver doing anything. Because that is a property of this particular build rather than of SQLite, the new test asserts the pragma directly as well as exercising each constraint and an ON DELETE CASCADE. Dropping rusqlite's bundled feature for a system SQLite would turn enforcement off, and this fails rather than letting the claim go quietly false. SQLForeignKeys is genuinely implemented over PRAGMA foreign_key_list, so an application that acts on the "Y" finds the metadata it then asks for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
connect now issues PRAGMA foreign_keys = ON, so that SQL_INTEGRITY = "Y" is true by construction rather than by build configuration. SQLite defaults the pragma off for backward compatibility. Enforcement worked only because the bundled library is compiled with SQLITE_DEFAULT_FOREIGN_KEYS, which is a property of one dependency's build and not of SQLite: dropping rusqlite's bundled feature for a system library would have turned referential integrity off while the driver went on advertising it. On the current build this changes no behaviour, which is the point -- the integration suite is unchanged at 46 passing. It removes the coupling between an advertised capability and a dependency's compile flags. The enforcement test now goes through Backend::connect rather than opening a rusqlite connection directly. A raw connection would only re-test the dependency's build configuration, which is exactly what this stops relying on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SQL_TIMEDATE_FUNCTIONS advertised SQL_FN_TD_CURRENT_DATE, _CURRENT_TIME and
_CURRENT_TIMESTAMP while the escape dialect deliberately did not translate
them, because SQLite spells all three as bare keywords and a name-only
remap cannot drop the trailing () the ODBC escape always carries.
The consequence was not a missing nicety: {fn CURRENT_DATE()} reached
SQLite as CURRENT_DATE(), which is a syntax error, so the driver advertised
three functions an application could not use. The module doc recorded the
limitation without following it through to the bitmap that promised them.
stackable-odbc-core's new EscapeDialect::rewrite_scalar_fn replaces the
whole escape, so a zero-argument call can emit a bare keyword. A call with
arguments declines the rewrite rather than discarding them: {fn
CURRENT_DATE(x)} has no SQLite spelling, and silently dropping x would be
worse than letting it fail.
The new FFI test executes each escape and checks the returned shape. It
fails without the rewrite, with the escape's parentheses reaching SQLite --
verified by reverting the dialect and re-running.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>stackable-odbc-core replaced its hand audit of default_get_info with a test that asks, of every info type, whether the answer moves when the backend does. Nine more values were statements about the data source that core had no way to know, and are now required Backend methods. Eight carry values this driver was already reporting, verified against the bundled library rather than moved on trust: column_alias, concat_null_behavior (SQL_CB_NULL -- 'a' || NULL is NULL), union_support, convert_functions, order_by_columns_in_select, accessible_tables, data_source_read_only and search_pattern_escape. SQL_SUBQUERIES changed. Core's default claimed SQL_SQ_QUANTIFIED, while this driver's SQL_SQL92_PREDICATES already excluded SQL_SP_QUANTIFIED_COMPARISON with a test recording that `< ALL`, `< ANY` and `< SOME` do not parse. The same capability was advertised by one info type and denied by another, and the advertised half is the one a BI tool acts on: it would push down a predicate SQLite rejects. The bit is dropped, and the new test probes all four claimed forms and all three unclaimed quantified spellings. accessible_tables is the one value here that describes the connected principal rather than the SQL dialect. "Y" guarantees SELECT on every table SQLTables returns, which is safe to claim only because SQLite has no principal and no per-table permissions -- opening the file is the whole access check. Also finishes SQL_KEYWORDS, which core answers with an empty string: that claims SQLite has no keywords of its own. It has AUTOINCREMENT, PRAGMA, VACUUM, GLOB, REGEXP and more, and applications read the value to decide what to quote. The list is read out of the linked library through sqlite3_keyword_count / sqlite3_keyword_name rather than transcribed from the documentation, so it describes the SQLite the driver links rather than the one an author was reading about, and filtered against the ODBC reserved list the spec defines this value as excluding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ownership table said core supplies the generic SQLGetInfo defaults. That stopped being true across four core releases: every value describing the data source is now a required Backend method, and leaving one unstated is a compile error rather than a silent inherited claim. An agent reading the old table would look for defaults that no longer exist. Adds a Declaring capabilities section carrying the three rules this work produced. Probe the bundled library rather than the documentation or the system sqlite3 -- they are different versions, and writing the ALTER TABLE bitmap from the CLI's behaviour got two bits wrong. Probe the bits not claimed as well, which is what caught them. And check that a new value agrees with the other info types describing the same capability, with a table of the five places this crate previously contradicted itself. Also records that connect issues PRAGMA foreign_keys = ON, and why the SQL_INTEGRITY claim depends on it rather than on a dependency's build flags. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven SQL_MAX_* values reported 0. The spec allows that for "no specified limit or the limit is unknown", and stackable-odbc-core answers it because it genuinely cannot know -- but SQLite enforces real limits, and an application reads these to decide whether to chunk a wide SELECT or a long IN list. 0 tells it there is nothing to chunk around. They are read from the connection through sqlite3_limit rather than hardcoded. That is not fastidiousness: sqlite3_limit both reads and writes, so a compile-time constant would be wrong for any connection that lowered one. The test proves the distinction by changing the limit and asserting the reported value follows -- comparing against SQLite's default would pass just as well against a hardcoded 2000. Enabling rusqlite's "limits" feature costs nothing; it is defined as an empty feature list and only unlocks the safe wrapper. SQL_MAX_TABLES_IN_SELECT keeps its 0. SQLite caps a join at 64 tables, but that is a compile-time constant with no sqlite3_limit behind it, and transcribing a documented constant is what produced the wrong ALTER TABLE bitmap earlier in this branch. Also zeroes SQL_MAX_CATALOG_NAME_LEN and SQL_MAX_SCHEMA_NAME_LEN, which inherited core's generic identifier length while this driver reports neither catalogs nor schemas -- a maximum length for a name that cannot exist. Both follow the support hooks rather than being pinned, so they stay correct if either flips. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
stackable-odbc-core added a required Backend::keywords and took ownership of ODBC_RESERVED_KEYWORDS and the filtering, so the spec's "excluding ODBC's own" rule is applied once across drivers instead of per backend. This driver now returns the raw list and deletes its own copy of the Appendix C table along with the subtract-sort-join it was doing. That table was a transcription of a specification page duplicated in every driver that needed it -- exactly the kind of shared fact core exists to hold. The list is still read out of the linked library rather than transcribed, and is still cached behind a OnceLock: core recomputes SQL_KEYWORDS on every call, because it cannot cache a value generic over the backend, and walking SQLite's keyword table each time would be wasteful. The test now asserts both halves, since each can fail alone. The raw list must still contain SELECT -- filtering is core's job, and doing it here too would reintroduce the duplication -- while the value reaching an application through get_info_raw must not, and must be shorter than the raw list, which is what proves the subtraction ran at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Core's error handling, type safety and trait surface all moved; this crate no longer compiled against it. The migration is one commit because no intermediate state builds. Error handling. `SqliteError` gains an `Odbc` variant and `From<OdbcError>`, which is what `Backend::Error`'s new bound requires and what makes the round trip through core lossless. Every `Backend` and `StatementBackend` method now returns `Self::Error`, removing the double conversion that stood at some thirty-six call sites in `metadata.rs`. The classified variants keep the `rusqlite::Error` they were classified from, so `SQLGetDiagRec` reports SQLite's extended result code through `NativeErrorPtr` and the diagnostic message carries the whole causal chain -- every error previously reached the application as native code 0. Capabilities. `identifier_case` is implemented and the `get_info_raw` arm answering the same thing is gone; `SQL_GETDATA_EXTENSIONS` goes back to core, which is where a fact about core's own fetch path belongs. `get_functions` is derived from `CORE_EXPORTED_FUNCTIONS` rather than hand-listed: the list had drifted to 53 of the 69 exported entry points. Nullability. With rusqlite's `column_metadata`, each result column reports what it actually is -- `SQL_NO_NULLS`, `SQL_NULLABLE`, or `SQL_NULLABLE_UNKNOWN` for a computed column, where SQLite reports no metadata and the driver genuinely cannot tell. Every column was previously claimed nullable. Tests. The 37 reach-ins into `ConnectionHandle` are replaced with helpers that drive setup and read-back through the FFI on their own statement handle, so they cannot disturb the state a test is asserting on. `handles` is `pub(crate)` in core now; the `test-support` feature carries `conformance`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`pre-commit` ran in its own workflow, so the `finished` job that branch protection keys on could not `needs:` it -- `needs:` cannot cross workflows. Formatting, clippy (which is what enforces the unwrap_used / unwrap_in_result / panic denies), cargo-deny and cargo-sort could all fail while the one required check went green. Moved the job into `build.yaml` and added it to `finished`. Also: a concurrency group that supersedes in-flight runs on the same ref, but never in a merge queue, where cancelling reports failure and evicts the PR; `timeout-minutes` on every job; `--locked` on the cargo-test, cargo-clippy and cargo-deny hooks, so CI cannot quietly resolve a dependency the lockfile does not name; and a cargo-doc hook, since broken intra-doc links are warnings that would otherwise reach a published doc build unnoticed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…on check Several project rules described a world that no longer exists. `handles` is `pub(crate)` in core, so the tests cannot reach `ConnectionHandle`; the FFI helpers that replaced that route are worth documenting, including why they allocate their own statement handle instead of borrowing the caller's. `close_cursor` is fallible. Backend and StatementBackend methods have one error type. Core re-exports `odbc-sys` wholesale, which retires the `RawTimestamp` mirror `src/ffi_integration_tests.rs` carried -- a hand-maintained `#[repr(C)]` duplicate of `SQL_TIMESTAMP_STRUCT` is one drift away from a silent ABI mismatch, and there is no longer any reason to keep one. Adds the fourth capability rule: declare it once. A `SQLGetInfo` value with a `Backend` hook must not also be answered in `get_info_raw`, because core derives the info type from the hook and the two can then disagree -- `SQL_IDENTIFIER_CASE` and `SQL_GETDATA_EXTENSIONS` were both stated twice. Also covers `SQL_ATTR_TXN_ISOLATION`, which core now validates against `txn_isolation_options`: a level this driver does not advertise is refused with HY024 rather than stored and echoed back. There was no test over that attribute at all; there is now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cargo test` ran in three places on every PR: the `unit-tests` job, the cargo-test pre-commit hook, and again inside `run-tests.sh` at the end of the integration job. Same command, same runner OS, no added coverage -- but two extra test-harness builds on two extra runners. Drops the `unit-tests` job, whose only distinguishing work was the duplicate `cargo test`; the downstream jobs now gate on `pre-commit`, which runs it via the hook. CLAUDE.md already points contributors at `pre-commit run --all-files` as the single source of truth for what must pass, and that is only true if CI runs the same thing rather than a hand-copied subset of it. Gives `run-tests.sh` a `--skip-cargo-test` flag, following the existing `--skip-build`, and passes it from CI. The default still runs everything, so a developer invoking the script by hand gets the whole suite in one command. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… for real Core moved twice: capability declarations became per-connection, and the catalog functions became typed rows core owns. Both are breaking, neither has a compiling intermediate state, so they land together with the cancellation work they made possible. Per-connection capabilities. The 25 required capability methods, `get_type_info` and `escape_dialect` take `&Self::Connection` — `SQLGetInfo` is a per-connection call. Every answer this driver gives is a property of the linked SQLite rather than of the file opened, so each ignores the argument, but `sqlite_get_info` now threads `Option<&SqliteConnection>` through to `default_get_info` and `common_get_info_raw`, and an arm consulting a capability hook has to be guarded on the connection being present. That is why `SQL_MAX_CATALOG_NAME_LEN` and `SQL_MAX_SCHEMA_NAME_LEN` only report 0 once a connection is open; pre-connect they fall through to core's generic identifier length, as every other `SQL_MAX_*_NAME_LEN` already did. Catalog result sets. The six catalog methods return typed row vectors instead of a `Self::Statement`; core converts them to the spec's column layout, sorts them, and serves the result set. So `metadata.rs` loses its descriptor construction, its `statistics_sort_key`, its per-table KEY_SEQ sort and the `SQL_ALL_*` discovery blocks — core detects those enumerations on the raw arguments and answers from `supports_catalogs`, `supports_schemas` and the new required `table_types` hook. Two defects fell out of the typed rows: `PKCOLUMN_NAME` is not `Option`, which surfaced that an implicit `REFERENCES parent` was reporting NULL in a column the spec marks "not NULL" (now resolved from the parent's primary key, per position), and core's new HY009 check surfaced that a null `SQLStatistics` TableName was answering `SQL_SUCCESS` with no rows. SQLCancel. `CancelToken` is `Arc<rusqlite::InterruptHandle>` and `cancel` calls `sqlite3_interrupt`, so a statement running on one thread can be stopped from another. This is the aliasing token shape core's doc names SQLite as the example of; the handle is captured in `connect` rather than fetched through the `Mutex`, because `cancel_token` can neither block nor fail, and `cancel` takes no lock this driver owns, which is what keeps SQLCancel's idle path — where core holds the connection's group lock across the call — from deadlocking. `map_sqlite_error` gains `ErrorCode::OperationInterrupted` -> HY008, without which a cancelled statement would report HY000. Also: `SQLITE_TYPE_INFO` is a `LazyLock` because `TypeInfoRow::new` is no longer `const`; `keywords`, `search_pattern_escape`, `get_functions`, `browse_connect_attrs` and `get_type_info` return `Cow`; `ColumnDescriptor`, `TypeInfoRow` and `EscapeDialect` are read through accessors; `SQL_ADD` and `SQL_DIAG_MESSAGE_TEXT` are gone in favour of the odbc-sys values. Tests. Two pinned defects core has since fixed, so both are rewritten rather than kept: `set_query_timeout_stored_and_retrieved` asserted that a 30-second timeout was stored and echoed back, and `sql_statistics_w_no_table_filter_also_succeeds` asserted the null-TableName success. `metadata.rs`'s unit tests move off `StatementBackend` onto the typed rows and assert only which rows exist and what each field holds; ordering is core's now, so it is asserted at the FFI level where core's sort has actually run. The cross-thread cancel test was verified by mutation: with `token.interrupt()` removed the query runs to completion (44s) and the test fails on the return code. Verified against stackable-odbc-core b047cb1: cargo test 274 passing, clippy clean, pre-commit green, and the pyodbc suite 23/23 DSN-less and 23/23 via DSN. Core has since landed 85a705a and 565f782, which change `Backend::tables` to take a parsed `&[String]` table-type list, so this does not build against core's current HEAD; that adaptation is deliberately left for a follow-up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ct four info values The follow-up the previous commit deferred, plus everything else core moved since. Three breaking changes with no compiling intermediate state, so they land together. Catalog query types. The six catalog methods take a sealed query object instead of five to eight positional arguments — `SQLForeignKeys` alone took six `Option<&str>` in a row, where crossing a primary-key argument with its foreign-key counterpart compiled without complaint. The query travels all the way into `metadata.rs` rather than being unpacked at the trait boundary, which would reintroduce that hazard one layer down. `TablesQuery::table_types()` is a `&[String]` core has already split on commas and stripped the quotes from, so `metadata::tables` loses its own parsing; a lone "%" still arrives, because the `SQL_ALL_TABLE_TYPES` enumeration core answers itself additionally requires the other three arguments to be empty strings, and is still read as no filter. Sealed rows. Every catalog row type is `#[non_exhaustive]`, so the eight struct literals become `Default` plus the consuming setter per column. A column this driver does not populate is now unnamed rather than spelled `None`, which is the point — it makes a column added to a spec result set a core-only change — so each site says in a comment which columns it leaves NULL and why. Ten new required capability hooks. Six are values this driver already stated and that now move out of `sqlite_get_info` into the hook, because answering in both places is the "declare it once" violation AGENTS.md describes: `driver_name`, `driver_version`, `dbms_name`, `dbms_version`, `integrity` and `txn_capable`. The snapshot pins all six regardless of which layer answers, which is what made moving them safe. Two more were already pinned at core's default and are now claims this driver makes on purpose: `accessible_procedures` "N" and `txn_capable` SQL_TC_DML. `driver_name`/`driver_version` take no connection — the Windows DM asks for driver identity before `SQLDriverConnectW` — while `dbms_name`/`dbms_version` describe what was connected to and take one. Two info values were wrong, and both are now live-probed rather than read off the documentation. `SQL_QUOTED_IDENTIFIER_CASE` claimed SQL_IC_SENSITIVE, telling an application that "T" and "t" are different tables; in SQLite double quotes are a delimiter, not a case-sensitivity switch, so it is SQL_IC_MIXED, and the probe asserts both halves of that — case-insensitive matching and mixed-case storage. `SQL_SPECIAL_CHARACTERS` claimed "", which was core's old default rather than a claim this driver ever made; SQLite parses `$` in an undelimited identifier, so it is "$", probed over 31 candidates with the rejected ones asserted too. The negative half is what stops it understating again, the same lesson `alter_table_capabilities_are_each_live_probed` records. SQLRowCount. Core now reads a zero-column statement reporting `Some(0)` as SQL_NO_DATA, per SQLExecDirect's Comments, which surfaced that this driver answered `Some(0)` for DDL — so every `CREATE TABLE` it ran returned SQL_NO_DATA to the application, and 60 of the 66 initial test failures were that. `row_count` now distinguishes "counted zero" from "no count applies". SQLite exposes no predicate for this (`sqlite3_stmt_readonly` is false for DDL too), so `is_searched_dml` decides from the leading keyword, past whitespace and both comment forms, counting REPLACE and WITH alongside the obvious three. That also removes a stale count: `sqlite3_changes()` reports the most recently completed INSERT, UPDATE or DELETE, so a `CREATE TABLE` run after a three-row INSERT was handed that 3 and reported it. Two values changed underneath us, both core-owned and both describing core's fetch path rather than SQLite, so the snapshot follows: `SQL_CURSOR_SENSITIVITY` to SQL_UNSPECIFIED, and `SQL_FORWARD_ONLY_CURSOR_ATTRIBUTES2` to SQL_CA2_READ_ONLY_CONCURRENCY. Tests. The two `get_info_every_named_info_type_has_the_declared_shape_*` tests asserted SQL_SUCCESS where core now documents that the shape probe's zero-length buffer is total truncation; they assert "not SQL_ERROR", which is what their own messages always claimed. `dbms_ver_is_well_formed` and `driver_ver_is_well_formed` read through the hooks, the first via `test_connection` since it needs a data source. `SQL_MULTIPLE_ACTIVE_TXN` has no `odbc_sys::InfoType` variant, so it is pinned through the raw path — the snapshot iterates named types only. Also fixes a pre-existing rustdoc failure that only surfaced once the crate compiled again: a public doc comment linked to the `pub(crate)` `SqliteConnection::interrupt`. Verified against stackable-odbc-core dd25a22: cargo test 280 passing, clippy clean, `pre-commit run --all-files` green across all 15 hooks, and the pyodbc suite 23/23 through real unixODBC. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…even stale claims
The README now follows the same shape as stackable-odbc-trino's: logo, badges,
Stackable links line, then what the thing is before how to build it. It is
written for a reader who does not already know what ODBC, a driver or SQLite
is, since that is who arrives at a driver repository. Two departures from the
Trino README, both because the alternative would be false here: it leads with
building from source rather than a releases page, there being no tags yet, and
every entry under Highlights names something the tests actually exercise.
The stale claims, all verified against the code rather than assumed:
- info.rs described MAX_FRACTIONAL_SECONDS_PRECISION as 0 in four places
("scale is fixed at 0", "'HH:MM:SS'", "column_size intentionally excludes a
fractional-seconds allowance"). It is 3, and the test asserting
default_precision_for_type(TIME) == 12 proves the fraction is budgeted.
- The C ABI entry points are 60 SQL* functions plus ConfigDSNW on Windows, not
73.
- The cancellation section's reason for having no query timeout, that the
synchronous execute path has no deadline to arm, is falsified by core's
query_timer.rs: QueryTimeout::CoreCancels arms one and calls Backend::cancel,
which this driver implements for real. The behaviour is unchanged, so the
text now records it as a gap with the preconditions for closing it.
- params.rs is a doc-only stub; binding is inline in execute.rs.
- CLAUDE.md's file sizes were off by up to 900 lines, and omitted backend.rs.
- The Windows "Add" button is not inert: core exports ConfigDSNW and both
installers register Setup=, so Add writes a DSN headlessly, silently missing
Database.
- Core's fetch benchmark moved to bench/benches/.
Prose double dashes and em dashes are recast as commas, colons, full stops or
parentheses throughout the comments and docs. backend.rs, execute.rs and
escape_dialect.rs already used one style and info.rs and the FFI tests the
other; now none of them use either. The SQL comments inside query literals and
the banner rules are untouched.
No CHANGELOG entry: nothing an application can observe changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>…red script paths `test/` and `windows/` were two top-level directories for one concern, and `test/` mixed four different things: the shell entry points, the SQL fixture, the pyodbc suite, and three generated files sitting in the same directory as the sources. The layout now matches stackable-odbc-trino's, minus everything that only exists because Trino needs a server: integration-tests/ setup.sh, run-tests.sh wrappers scripts/ lib.sh + the logic suites/ create_test_db.sql, test_integration.py generated/ gitignored wholesale windows/ WINDOWS.md, windows_test.py, vm/ There is no stack/, no compose file and no scripts/teardown.sh: SQLite is a file and rusqlite links its own copy, so there is nothing to stand up or tear down. That is also why this suite gates every pull request while Trino's cannot. The script cleanup: - Both scripts derived PROJECT_DIR, DRIVER_PATH and DB_PATH separately, and disagreed about which directory DB_PATH lived in relative to the script. They now source scripts/lib.sh, which owns those and the ODBCSYSINI/ODBCINI export, the driver build, and a setup precondition check. - Generated output moves out of the source directory into generated/, ignored by a `*` .gitignore rather than by naming each file. All of it embeds absolute paths, so a committed copy is wrong for everyone but its author. - run-tests.sh forwarded any unrecognised argument to windows_test.py even without --windows, so a typo'd flag produced a full green run that had ignored it. Unknown arguments are now rejected unless --windows is given. - run-tests.sh required setup.sh to have been run and said so nowhere; it now checks and points at setup.sh instead of failing inside pyodbc. - Both take --help, printed from the header comment block by lib.sh's usage() rather than a hardcoded line range that truncates as soon as a line is added. - setup.sh gained --skip-build, and rejects unknown arguments. shellcheck now runs with -x so it follows lib.sh, the same reason the Trino repository passes it; without it every sourcing script reports SC1091 for a file that is right there. Also fixes two path references that were already stale before the move: windows/vm/start.yaml pointed at `test/sqlite/windows_test.py`, a path this repository has never had, and test_integration.py's usage line named itself under test/. Verified by running setup.sh and run-tests.sh: 23 pyodbc tests pass DSN-less and again through the DSN, the require_setup guard fires with the database removed, and unknown arguments exit 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…iver's level Four changes that arrived together, kept in one commit because the packaging half is what makes the driver half reachable. **The setup dialog.** Core's Backend::configure_dsn was defaulted to the identity function, so this driver wrote a data source with no Database key whenever the ODBC Administrator's Add... button was pressed. Core owns all of ConfigDSN -- validating the request, merging the data source's stored keywords in, calling SQLValidDSN and writing through SQLWriteDSNToIni. src/backend/setup.rs supplies only the dialog, and the dialog itself is packaging/windows/configure-dsn.ps1 run with -Emit, which prints the keywords it collected instead of writing them. Reusing the script keeps one list of keywords: its $Fields table names them, and dsn_keys_match_the_connection_string_parser fails the build if that table and the parser disagree. Only the two kernel32 calls are cfg(windows); every decision is a plain function unit-tested on Linux. The dialog is a fifth of the Trino driver's, because SQLite has one connection-string keyword. Its Test button reports the SQLite version *and the table count*, which is the check worth having here: SQLite creates a missing file rather than refusing, so a typo in the path connects perfectly well and finds nothing. **The version resource.** build.rs embeds a Windows VERSIONINFO resource, without which the Administrator lists the driver as "Not marked" under Version and Company, as it does for every Rust cdylib. Every string comes from Cargo.toml through cargo's own environment, so the resource cannot disagree with the package. **Core from git.** Taken from the repository rather than a sibling checkout, so a clean clone and CI can build without one. deny.toml allows that one repository by name; CONTRIBUTING.md documents the [patch] override for working on core at the same time, which is the common case here. **The SBOM pipeline.** Both archives now carry a CycloneDX SBOM generated from the dependency list cargo-auditable embeds in the binary rather than from Cargo.toml, so it describes what was linked. sbom-native.json declares the two components cargo cannot see, verified against the real binaries by `sbom.sh --check-native` on every pull request: SQLite itself, which cargo sees only as the libsqlite3-sys wrapper, and what each artifact links at load time. The release also carries SPDX, sha256sums.txt and build-provenance attestations. CI gains a Windows unit-test job -- the only one that compiles backend::setup's cfg(windows) half -- a release-artifacts job running the native checks, a Scorecard workflow, and a `finished` gate derived from needs.* rather than a hand-written list. Enabling --document-private-items on the cargo-doc hook surfaced two intra-doc links that had been broken and unchecked; both are fixed here. Verified on the Windows VM against the real Administrator: Add... opens the dialog with an editable name and no scope radios, Test connection reports SQLite 3.53.2 and the fixture's three tables, OK writes the data source, Configure... prefills it and fixes the name as the spec requires, Cancel leaves it untouched, Remove takes it without prompting and without touching the database file, and -NoGui writes headlessly. The Drivers tab lists the driver as 0.00.01.00 / Stackable GmbH. The 46 integration tests through the Windows Driver Manager still pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…le claims The docs described a driver that no longer matches the code in five places, and read as development history rather than as reference material. Corrected: - AGENTS.md called core a path dependency with a matching `TODO`, and said CI could not pass because of it. Core is a git dependency, there is no `TODO`, and CONTRIBUTING.md already documented the real `[patch]` mechanism. - WINDOWS.md still described the driver as headless with no setup dialog, which the previous commit added. It also told the reader to use the DSN `test_sqlite` and then created `MySQLite` against another database. - A 54-line doc block in `info.rs` had no blank line after it, so it attached to `SQLITE_SUBQUERIES` and left `SQLITE_ALTER_TABLE` undocumented. - The README's `isql` example named `test/test.db`, which does not exist, and a release build, where `setup.sh` produces a debug one. - Four files claimed all three generated files embed absolute paths. Only `odbc.ini` and `odbcinst.ini` do; `test.db` is a binary. Two flags `windows_test.py` accepts, `--target` and `--vm-network`, were missing from the docs and from the `--help` text `lib.sh` derives from the run-tests.sh header. Restructured: - CHANGELOG.md becomes a first-release capability statement, as the Trino driver's does, replacing 414 lines of development history. - README.md follows the Trino driver's order and gains Compatibility, Troubleshooting and Getting help. Testing and Releasing move to CONTRIBUTING.md, and the shared-core framing moves with them. - AGENTS.md leads with the architecture rather than reaching it 61% in, and the ODBC design rationale gets its own section instead of sitting under Conventions. The module table gains `setup.rs`, `build.rs` and `benches/`, and a section covers the setup dialog and the test that pins it. - CLAUDE.md links into AGENTS.md for the six rules it restated verbatim. - packaging/README.md puts support and the SBOM ahead of the build instructions, since it ships inside the archive. Prose: em dashes drop from 31 to 1, the survivor being the changelog heading format in release.toml, which the Trino driver shares. Comments that narrated what core used to default to are now present-tense statements of the rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The driver reported `SQL_TC_DML`, which the spec defines as "Transactions support only Data Manipulation Language (DML) statements [...] Data Definition Language (DDL) statements encountered in a transaction cause an error." SQLite does the opposite. Measured against the bundled 3.53.2, a `CREATE TABLE` between two inserts inside a transaction raises nothing, and a later `ROLLBACK` undoes the table along with the rows. That is `SQL_TC_ALL`, "Transactions support both DML and DDL statements in any order". The hook's own doc comment already described the correct behaviour and then picked the contradicting value, on the grounds that `SQL_TC_DML` was the weaker, safer claim. It is not a weaker claim, it is the opposite one: an application reading it before running DDL inside a transaction either refuses, or commits first and silently discards the atomicity the user asked for. `transaction_capability_is_live_probed` now measures it, and separates all four non-`NONE` values in one run: no error rules out `SQL_TC_DML`, the surrounding inserts disappearing rules out `SQL_TC_DDL_COMMIT`, and the created table disappearing rules out `SQL_TC_DDL_IGNORE`. Each assertion names the value it eliminates, so a future SQLite change points at the right answer. No changelog entry: nothing has been released yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`set_query_timeout` fell through to core's default, so a requested timeout was substituted with `0` and reported as `01S02`: an application asking for thirty seconds was told, correctly but unhelpfully, that it had no deadline at all. SQLite has no server-side statement deadline to set, so `QueryTimeout:: DataSource` is unavailable. `QueryTimeout::CoreCancels` is the honest answer and its precondition already held: `cancel` really cancels, via `sqlite3_interrupt`. Core now arms its own timer and calls it when the deadline passes. The deadline covers execution rather than fetching, which is where the time goes, because `exec_direct` materialises every row before returning. `CoreCancels` asks for `is_cancelled` alongside, which needed a cancel token that can be observed. `Backend::CancelToken` becomes `SqliteCancelToken`, pairing the connection's interrupt handle with a flag of its own. The flag is minted fresh per `cancel_token` call rather than shared with the connection: core mints a token per statement-producing call, and a shared flag would leave a cancelled statement permanently unusable, where the spec says "After the statement has been canceled, the application can call SQLExecute or SQLExecDirect again." `query_timeout_stops_a_long_running_statement` runs a recursive CTE past a one-second deadline through the real entry points. Verified by mutation in both directions, which corrected an assumption worth recording: `HYT00` does *not* come from `is_cancelled`. Core marks its own `CancelState` timed out before cancelling and relabels the failure ahead of the `HY008` reclassification, so stubbing `is_cancelled` to `false` leaves the test passing while reverting `set_query_timeout` fails it. AGENTS.md claimed the opposite and is corrected. The README and the changelog's capability statement both listed the missing timeout as a limitation; both now describe what the driver does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite here went through pyodbc, so unixODBC answered a large part of the ODBC state machine before the driver ever saw it. What this driver does with an out-of-order or malformed call was untested. `test_c_abi.py` loads the `.so` with ctypes and calls the exported entry points with no Driver Manager in the loop: handle lifecycle and parentage, stale handles, double frees, use after free, cursor state, prepare/execute/re-execute, SQLFreeStmt options, the 01S02 attribute substitutions and their read-back, the enforced query timeout firing and the statement staying usable afterwards, transactions with DDL inside them, and the four catalog functions SQLite answers with an empty result set of the right shape. 120 probes. Where the spec attributes a SQLSTATE to the Driver Manager, nothing produces it here, so those probes assert what the driver does instead and name the (DM) diagnostic they are not demanding. `harness.py` and `odbc_abi.py` are the shared machinery, ported from the Trino driver. `Stack` is replaced by `Target`, which parses the connection string the suites already take, because SQLite needs no running stack to describe. Writing it turned up two things. The driver was right and the first draft was wrong about `SQLColumnPrivileges`: its `TableName` "cannot be a null pointer" per the spec, and the driver already answers `HY009` with a message that says so, which is now asserted rather than tripped over. The second is a real bug. `--skip-build` added itself to the forwarded-argument array so it would also reach `windows_test.py`, but the guard that rejects Windows-only flags on a Linux run counted it, so the documented invocation `run-tests.sh --skip-build` always failed. It is now acted on locally and appended to the forwarded arguments only when `--windows` is also passed, so the guard still catches a typo'd flag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…inding Manual-commit transactions had FFI-level coverage but nothing exercising them through a Driver Manager, where an application meets them. Three of the eight scenarios assert the opposite of the Trino driver's, which is why the suite could not be copied across: - A failed statement leaves a SQLite transaction usable. Trino aborts the whole thing and refuses the commit, so its driver rolls back and reports 25S03. Here the commit must succeed and publish the earlier writes; a driver that rolled back to look consistent would discard writes silently. - A commit preserves an open cursor. `SQL_CURSOR_COMMIT_BEHAVIOR` is `SQL_CB_PRESERVE`, true only because `exec_direct` materialises eagerly, so the scenario fetches on after the commit and expects the rest of the rows. - Serializable is the level that must be accepted, and the other three refused with HY024. Trino is the mirror image. Both halves are asserted: refusing everything would pass a rejection-only check while offering no isolation at all. Also covered: rollback and commit visibility from a second connection, atomicity across two tables, autocommit as the default, and DDL inside a transaction being undone by a rollback, which is `SQL_TC_ALL` seen from the application's side rather than through the C ABI. Writing it surfaced a finding that is not fixed here. `count(*)` comes back as a *string*: `sqlite3_column_decltype` is NULL for any computed column, and `describe_column` falls back to `TEXT`, so every expression is described as VARCHAR whatever the storage class of its value. Aggregates are a common enough shape that a BI tool would see text where it expects a number. The suite coerces with a documented `as_int` so a transaction failure is never reported as a typing failure, and the fix belongs with the type-matrix suite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…m fuzz `SELECT count(*)` was described as `SQL_WVARCHAR`. So was every other computed column: a literal, an expression, an aggregate, and even an explicit `CAST(x AS INTEGER)`. `sqlite3_column_decltype` names the column of a stored table or nothing at all, and `describe_column` fell back to `TEXT` whenever it answered nothing. Aggregates are among the most common shapes a BI tool sends, and a text column is one it will not offer to sum or chart, so this was visible in exactly the place the driver is meant to be useful. `infer_decl_type` supplies the missing declaration from the storage classes of the materialised values. It returns a *string* rather than a `SqlDataType`, so precision, scale and `SQL_DESC_TYPE_NAME` all come from the same functions that handle a real declaration and an inferred `INTEGER` column is indistinguishable from a declared one. Mixed classes resolve to whatever holds every value present: integers and reals to `REAL`, anything with text to `TEXT`. NULLs are skipped, because a NULL is the absence of a value rather than evidence of a type, and counting one would make the description depend on which rows matched. No rows, or nothing but NULL, keeps the old `TEXT` fallback. A declared type still wins over the values. SQLite lets any value into any column, so an `INTEGER` column can hold text; the declaration is what the schema promises and what the next row might hold. Both statement paths now collect rows before building descriptors. Values are converted using the descriptor's SQL type, so refining it afterwards would convert against the old one. Nothing is read twice: the rows are materialised either way. `test_type_matrix.py` was written first and failed on ten of these. It drives every (value, C type) pair through `SQLGetData` against invariants rather than a transcribed conversion matrix, then checks what `SQLDescribeCol` reports, which is the separate question an application asks first. 298 checks. The transactions suite loses the `as_int` workaround it carried for this, and its `count(*)` comparison against an integer now stands as a regression guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t time
Walks the SQL a BI tool emits through the Driver Manager: joins of every shape,
aggregates, window functions, subqueries and CTEs, set operations, and
parameters in every clause that takes one. 80 checks, run once per connection
style.
Two things it covers that the Trino driver's equivalent cannot.
`escape_dialect.rs` had no integration coverage at all. The suite drives
`{fn ...}`, `{d ...}`, `{t ...}`, `{ts ...}` and `{oj ...}`, including the three
date/time forms that are bare keywords in SQLite: `SELECT CURRENT_DATE();` is a
syntax error, so a name swap cannot express them and `rewrite_scalar_fn`
replaces the whole escape instead. That path is now exercised end to end.
And SQLite actually publishes keys and indexes, where Trino publishes none. The
fixture carries a primary key, a foreign key with a named target column and an
index, so `SQLPrimaryKeys`, `SQLForeignKeys`, `SQLStatistics` and
`SQLSpecialColumns` are asserted on real rows rather than on an empty set that
did not error. The foreign-key check pins `PKCOLUMN_NAME`, which the spec marks
not-NULL.
Two probes assert a refusal rather than a result, for the capabilities the
driver deliberately does not claim: `GROUP BY GROUPING SETS`, absent from
`SQL_GROUP_BY`, and `> ALL`, absent from both `SQL_SUBQUERIES` and
`SQL_SQL92_PREDICATES`. Each names the token SQLite must complain about, so a
statement failing for an unrelated reason cannot pass them, and a control query
of the same CTE shape confirms the syntax around them is sound.
The inline relations are CTEs with column lists rather than Trino's
`(VALUES ...) AS t(x)`, which SQLite has no syntax for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>Asks the shapes `test_sql_surface.py` covers, at a size nothing else reaches:
multi-table joins, UNIONs, subqueries, CTEs, window functions, 50,000-row
fetches, wide rows, and a prepared statement re-executed two hundred times.
Size is the point rather than a detail. `exec_direct` materialises every row
before returning, so a large result set is where that decision is felt, and the
suites so far all work in tens of rows.
The fixture is generated by recursive CTE rather than shipped, so the repository
carries no data file and the size is three constants at the top of the module.
It lives in a database of its own, `generated/stress.db`, so the shared one
keeps the size the other suites expect and a half-finished stress run cannot
affect them.
One scenario needed rewriting rather than porting: SQLite refuses a LIMIT on an
operand of a compound SELECT ("LIMIT clause should come after UNION ALL not
before") where Trino accepts it, so each branch is wrapped in a subquery to make
the LIMIT bind per branch.
No counterpart to the Trino driver's `parse_profile.py` and `profile_stress.sh`
is included, deliberately. Those attribute a query's time between the
coordinator and the client, and SQLite has no server-side half: the query runs
in-process inside the same shared object. `cargo bench` already measures fetch
throughput, which is the question that remains here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>…on Windows Two fixes and one gap. `test_integration.py` created `types_test` with `IF NOT EXISTS` and dropped it at the end, so it removed a table `create_test_db.sql` owns and left whatever ran next to find it missing. Its fixtures are now named `it_*`, created and dropped by this suite alone. That is what made the shared database look corrupted mid-session earlier. It also predated `harness.py` and carried its own `run()` and PASS/FAIL counters, so it was the one suite whose output did not match the rest. It now uses `Results` and `Target` like the others. Three stale coercions went with that: `int(count)` around `COUNT(*)`, and a comment saying it "may come back as str depending on column type metadata". That was true until computed columns started being typed from their values, and papering over it now would hide a regression rather than tolerate a known one. Every test is kept, including the ones `test_sql_surface.py` duplicates, because until this commit `test_integration.py` was the only suite the Windows VM ran and its breadth was the whole of Windows coverage. Three of them are unique anywhere: `SQLRowCount` for INSERT/UPDATE/DELETE, Unicode round-tripping through core's UTF-16 marshalling, and the `SQLGetData` path that a pyodbc output converter forces in place of `SQLBindCol`. That "only suite the VM runs" was a deploy-list limitation rather than a real constraint, so it is gone too. `windows_test.py` now ships `harness.py`, `odbc_abi.py` and all six suites, and runs them the way the Linux runner does: the pyodbc ones per connection style, the ctypes ones once, the stress suite once against its own database. It records every result instead of stopping at the first failure, since one Windows-only defect should not hide the next and a VM round trip is expensive enough that a second run to find out is a real cost. The Windows half is unverified: this machine has no VM. The Linux run is green across all six suites. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… Windows `test_stress.py` goes through pyodbc like the other Driver Manager suites, but it was handed the DLL path that the two ctypes suites need. unixODBC resolves `Driver=` as either a name or a path; the Windows Driver Manager looks it up in ODBCINST.INI only, and answered IM002. Verified on the VM: all six suites now pass on Windows, in the same shapes and with the same counts as on Linux. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eleven commits, of which five are fixes an application can observe: a float narrowing that overflowed wrote a wrong value instead of reporting 22003; a float target reported diagnostics its spec row does not define; SQLGetDescRecW counted its buffer and length in bytes where the spec counts characters; MAX_NTS_SCAN refused long SQL_NTS statements that every other driver executes; and undocumented off-table SQLSTATEs now trip the diagnostics guard. The rest are documentation. The lock was still pinned to 5bc0417, so none of it was reaching this driver. Verified after the bump: cargo test green, both Linux integration runs green, and all six suites green on the Windows VM. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Spec (SQLGetData, Diagnostics): 07009 "Invalid descriptor index" covers a column number "greater than the number of columns in the result set". That clause carries no (DM) marker, so the Driver Manager does not supply it and the driver has to. Core does not range-check the ordinal itself. Its doc comment says the check is "delegated to the backend", so whatever SQLSTATE `SqliteStatement::get_data` chooses is what the application sees, and it chose `SqlState::general_error()`. An application asking for column 99 of a two-column result was told HY000, which says only that something went wrong and nothing about which argument was wrong. Confirmed through unixODBC as well as against the raw entry point: the Driver Manager passes it straight through. The column-0 arm gets the same SQLSTATE. It is unreachable through SQLGetData, because core rejects the bookmark ordinal before calling, but leaving it as a general error would mean the two ways of naming a column that does not exist disagreed depending on which layer caught it. `describe_col`'s equivalent arms are deliberately left alone. Core range-checks that hook against `StatementBackend::column_count` before calling it and documents HY000 as the right answer for a genuine failure there, so those arms are unreachable defensive code rather than a second instance of this bug. The regression test probes one past the last column and u16::MAX -- asserting only the first would pass for an implementation that special-cased count + 1 -- and reads a real column afterwards, so it cannot be satisfied by refusing every ordinal. Checked to fail without the fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… hostile catalog arguments
Three gaps a pen test of the driver turned up, none of which any suite reached.
`test_c_abi.py` gains the SQLGetData buffer contract, which is the part of that
call an application cannot avoid and a Driver Manager does not implement:
exact fit, one byte short, the indicator reporting the untruncated length, the
zero-length length probe writing nothing, a negative BufferLength refused with
HY090, a wide indicator counted in bytes rather than characters, and the 07009
ordinal check the previous commit fixed. Buffers are guard-filled before each
call, so a write past the length the driver was given is visible rather than
landing in memory that happened to be zero.
The one worth naming is chunked retrieval: a second SQLGetData call has to
continue the value, not restart it. A driver that restarts turns the documented
drain loop into an infinite one, and no amount of correct data compensates for
that. It is asserted by reassembling 26 characters out of ten-byte reads and
requiring the loop to end in NO_DATA.
`test_sql_surface.py` gains the other half of escape translation. Everything
there proved the rewriter fires; nothing proved it stops. A {fn ...} inside a
string literal, inside either comment form, or inside any of SQLite's three
identifier-quoting styles has to survive verbatim, because a rewrite there
changes the value a query returns with no error anywhere -- corruption rather
than failure. A doubled quote is included, since mishandling '' ends the
literal early and rewrites the rest.
It also gains the scalar-function bitmaps as a contract. A SQL_*_FUNCTIONS bit
is a promise: a BI tool emits {fn NAME(...)} only for the bits the driver sets,
so a set bit whose escape does not execute is a query the tool will build and
the driver will reject. Every bit in all four bitmaps is read back from
SQLGetInfo and the matching call executed with spec-shaped arguments -- which
is what would catch a name mapped to a SQLite function of a different signature,
the reason LOCATE is deliberately absent. Nothing else tied `info.rs`'s bitmaps
to `escape_dialect.rs`'s remap table. All 22 currently advertised functions pass.
Last, the catalog functions are given hostile names. They are the only path in
the driver that turns a caller-supplied argument into SQL, and for a BI tool
that argument is often typed into a filter box. The fixture grows a table whose
name contains a single quote, which can only be found if the arguments are
bound rather than interpolated. The injection payloads close a literal and
issue a DROP against the fixture, and the check is that the fixture is still
standing afterwards: "no exception" would also pass for a driver that ran them.
`%` and `_` are asserted to still work as patterns, which rules out satisfying
all of it by escaping everything indiscriminately.
CHANGELOG records one limitation the pen test established rather than fixed:
SQL_C_NUMERIC cannot be used to retrieve a value. A DECIMAL column is described
as SQL_DECIMAL and reads correctly as SQL_C_CHAR or SQL_C_DOUBLE, but
SQLGetData and a bound column both report 07006, while the same type works as a
parameter. That lives in core, which has no arm for it in `write_column_value`.
Linux is green: 125 in the SQL surface suite, 132 in the C ABI one, and every
other suite unchanged. The new checks are unverified on Windows -- the VM was
shut down before they were written -- though nothing in them is
platform-specific.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>Nine commits, none of which touch `src/backend.rs`: the `Backend` and `StatementBackend` traits, their signatures, their defaults and the capability hooks are unchanged, so this driver adopts nothing new and overrides nothing it did not already. What changes sits under the FFI boundary. `SQLGetData` now converts a chunkable value once and drains it across calls instead of re-materialising it per call, taking an N-byte column read through a K-byte buffer from O(N^2/K) to O(N); `sql_fetch` builds its binding list once, in column order; and the string fetch conversions stop allocating per value. Here that is the `TEXT` and `BLOB` path, where an application reading a large value through whatever buffer size a driver manager picks paid the amplification. The rest is test and documentation work in core: misaligned-buffer coverage for every marshalling family, error-path tests asserting the SQLSTATE their names claim, and the `Backend::connect` failure path. `cargo test` (296 passed) and `pre-commit run --all-files` (all sixteen hooks) pass on the new pin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"Verify DLL exports" could not fail. It piped a symbol count into `xargs
-I{} echo`, and a step's exit status is the last command in the pipeline,
so the grade came from `xargs`, which returns 0 whatever it is handed.
The job's own comment claims it "checks ... that the DLL exports the ODBC
entry points"; measured against an empty file standing in for a DLL that
exports nothing, the old form printed "SQLite DLL: 0 ODBC symbols
exported" and exited 0.
That matters because this is the only check on the artifact users load.
A `forward_ffi!` regression, a linker script change or a build.rs edit
that stopped exporting the entry points would ship, and the first symptom
is the driver failing to load with no diagnosis, which is what
README.md's troubleshooting section already fields.
It now reads the export address table and asserts 23 named entry points
-- the ones an application actually reaches the driver through, plus
`ConfigDSNW`, which the ODBC Administrator's "Configure..." button needs
-- and a floor of 55, so a wholesale regression is caught even if those
particular names survive. Measured locally: the DLL exports 61 and the
Linux .so 60, the difference being `ConfigDSNW`, which is
`#[cfg(windows)]`. That matches AGENTS.md's "60 `SQL*` functions, plus
`ConfigDSNW` on Windows".
Verified in both directions before landing: exit 1 against a DLL
exporting nothing, exit 0 against the real cross-built DLL.
The same defect and the same fix are in stackable-odbc-trino, where a
pre-release review found it first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>Twenty-four commits. One touches `src/backend.rs`: `StatementBackend::take_value_warning`, defaulted to `None` and drained by core after every `get_data`, through which a backend raises `01S07` for fractional precision it dropped inside its own type conversion, before a `ColumnValue` existed and so where core cannot see it. This driver adopts nothing there, because it never drops one. A `TEXT` datetime reaches core as `ColumnValue::String` and core parses it; the `INTEGER` epoch-seconds encoding has no sub-second part to lose; and `decode_julian_day` rounds an `f64` to the nearest nanosecond, which is the closest representation of a value that never held finer precision rather than a digit discarded. The declared-versus-delivered gap this driver does have runs the other way: `MAX_FRACTIONAL_SECONDS_PRECISION` reports 3 while the Julian-day path can deliver 9, and `01S07` names the opposite case. Where core drops a fraction itself, a `ColumnValue::Time` fraction written to `SQL_C_TYPE_TIME` or a fraction lost reaching an exact-integer C type, core raises the record, and the new method's doc is explicit that a backend must not report it a second time. The change that does reach this driver is `SQLGetData` range-checking the column ordinal in core, against `StatementBackend::column_count`. `SqliteStatement::get_data` answers `07009` for the same condition and is unchanged, but core now catches it first, so that arm is no longer what an application sees. Two doc comments said core delegated the check and are corrected; the arm itself stays, so a direct backend call and the FFI path cannot disagree about what a column that does not exist is called. `get_data_column_past_the_last_is_invalid_descriptor_index` passes either way, which is what makes it worth keeping. The rest is inert here. `SQL_C_GUID` and `SQL_C_NUMERIC` as retrieval targets, the SQL-to-C interval tables and `SQL_C_DEFAULT` selecting `SQL_C_GUID` all need a declared type this driver never produces: `sqlite_type_to_sql_data_type` maps `NUMERIC` to `DECIMAL` and has no GUID or interval arm at all. The parameter fixes (`HY090` for an undefined negative indicator, refusing a C type core cannot marshal rather than binding NULL), `SQLColAttributeW`'s `HY091`, the ODBC 2.x datetime codes in a descriptor and the float display size are all core's FFI layer, below the `Backend` boundary. `pre-commit run --all-files` passes all sixteen hooks (296 unit tests). Both integration suites pass on the new pin: 769 assertions through unixODBC on Linux, and the same 769 through the real Windows Driver Manager in the VM. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An em dash pair breaks the reading flow of a sentence that a comma or a real parenthesis carries without the interruption. Four passages, none of which change what they say: - `AGENTS.md`, the `HY008` clause, to a comma. - `integration-tests/README.md`, the hostile-catalog-argument aside and the drain-loop aside, both to parentheses, which is what the second pair was doing anyway. - `test_c_abi.py`, where the dash introduced the list of what the `SQLGetData` buffer contract covers. Parentheses, because the sentence already opens with "Covers:" and a second colon would read as a second list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Core is public now and carries a v0.1.0 tag, so pin the tag rather than the scaffolding branch. A tag resolves to one immutable commit, which a branch does not, and the SBOM's purl already names the resolved commit either way. Also drop the release.toml rule that rewrote the linux archive name in packaging/README.md. That file names archives with a `<version>` placeholder because it documents a naming scheme rather than one release, so the rule matched nothing and its `exactly = 1` aborted `cargo release` before it could tag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Extracts
stackable-odbc-sqlitefrom thestackable-odbc-rsworkspace, where itwas developed alongside the core framework, into this repository. Mirrors the
layout and commit sequence the Trino split settled on.
stackable-odbc-coreis a path dependency on a sibling checkout until it ispublished. CI cannot pass until that resolves — a path dependency does not
resolve on a runner. There is a matching
TODOinCargo.toml.The move
Sources, benchmarks, integration tests, the Windows VM harness and the release
packaging move across; the manifest is rewritten to stand alone, replacing
workspace inheritance with explicit values and adding the package metadata the
workspace crate never carried. Version is
0.0.1, matching core and the Trinodriver.
The first commit deliberately does not compile. It is the relocation and nothing
else, so it can be reviewed as one; the second carries the entire behavioural
delta from core's newer API.
Deliberately not carried over: the fuzz targets and the Miri job, which test
core's code and moved with it, and
SQL_SC_SQL92_ENTRY-era assumptions (below).CI keeps the SQLite integration suite, which the Trino repository had to drop —
this one needs only unixODBC and the
sqlite3CLI.Conformance fixes
Auditing all 64
SQLGetInfovalues against the specification, and adapting tofour rounds of core making invented values into required
Backendmethods,turned up a consistent defect: one capability stated twice, in opposite
directions, with nothing tying the two together.
SQL_CATALOG_NAME = "N"SQL_CATALOG_TERM = "catalog",SQL_CATALOG_NAME_SEPARATOR = "."SQL_OUTER_JOINS = "Y"SQL_OUTER_JOIN_CAPABILITIES = 0SQL_SQL_CONFORMANCE = SQL_SC_SQL92_ENTRYSQL_GROUP_BY = SQL_GB_NO_RELATIOSQL_SQL92_PREDICATESwithout quantified comparisonSQL_SUBQUERIESwithSQL_SQ_QUANTIFIEDSQL_TXN_ISOLATION_OPTIONwith four levelsOthers were plain misstatements:
SQL_CURSOR_COMMIT_BEHAVIORreportedSQL_CB_DELETEwhile preserving cursors;SQL_ALTER_TABLEandSQL_OUTER_JOIN_CAPABILITIESreported0;SQL_INTEGRITYreported"N"despite SQLite enforcing the whole Integrity Enhancement Facility;
{fn CURRENT_DATE()}was advertised but reached SQLite asCURRENT_DATE(), asyntax error.
Each is a
CHANGELOG.mdentry with its reasoning.Two behaviour changes worth review
connectnow issuesPRAGMA foreign_keys = ON. Enforcement previouslyworked only because the bundled library compiles with
SQLITE_DEFAULT_FOREIGN_KEYS— a dependency's build flag, not a property ofSQLite. No behaviour change on the current build; it stops referential
integrity turning itself off if that dependency changes.
SQL_SQL_CONFORMANCEreports0, claiming no SQL-92 level. The previousSQL_SC_SQL92_ENTRYwas a core default, never an assessment of SQLite, andentry level requires
SQL_GB_GROUP_BY_EQUALS_SELECT— which SQLite fails.Honest, but some tools gate on a conformance claim.
How capability values are established
Probed against the linked library, never transcribed. This is not
fastidiousness: writing the
ALTER TABLEbitmap from the systemsqlite3(3.51.3) got two bits wrong, because the bundled library is 3.53.2 and accepts
ADD CONSTRAINT/DROP CONSTRAINT. The probes assert unclaimed bits arerejected as well as claimed bits accepted — that negative half is what caught
it.
SQL_KEYWORDSand theSQL_MAX_*limits go further and read their valuesout of the library at runtime, so they need no maintenance.
AGENTS.mdrecords the rules under Declaring capabilities.Verification
263 unit and FFI tests; 46 Linux integration tests through real unixODBC; 46
Windows VM tests over WinRM (DSN-less and via DSN);
fmt,clippy --all-targets --all-features -D warnings,doc,denyand the fullpre-commitgate clean; Windows DLL cross-compiles and exports 76 ODBC symbols.