Uh oh!
There was an error while loading. Please reload this page.
FIX: gate SQL_WVARCHAR output-converter fallback to string/binary columns (#691) - #692
Conversation
…umns (#691) Cursor._build_converter_map applied a converter registered for SQL_WVARCHAR as an unconditional catch-all to every column that lacked a direct type-keyed converter, so a lone SQL_WVARCHAR converter mangled INT / DECIMAL / DATE values. Gate the fallback on the column's mapped Python type being str/bytes, mirroring the isinstance(value, (str, bytes)) guard already used in Row._apply_output_converters, so the optimized and fallback apply paths agree. Also drop the now-redundant in-loop import of ConstantsDDBC (already imported at module scope as ddbc_sql_const). Adds a regression test that spies on converter invocations and asserts the converter fires only on the NVARCHAR / VARBINARY columns (a call-count contract). This is necessary because the optimized apply path swallows converter exceptions: a plain value.decode() converter raises AttributeError on int/Decimal/date and is silently swallowed, so a value-only assertion would pass with and without the fix. The rewritten test fails on the unfixed code and passes once the fallback is gated.
There was a problem hiding this comment.
Pull request overview
This PR fixes an output-converter dispatch bug in the optimized fetch path: a converter registered for SQL_WVARCHAR was being treated as an unconditional fallback for any column without a direct converter, which could corrupt non-string types (e.g., INT, DECIMAL, DATE). The change gates the legacy SQL_WVARCHAR fallback so it only applies to columns whose mapped Python type is str or bytes, aligning the optimized converter map path with the existing runtime guard behavior.
Changes:
- Gate the
SQL_WVARCHARfallback inCursor._build_converter_map()to only apply whendesc[1]maps tostr/bytes. - Add a regression test covering mixed
INT/DECIMAL/DATEwithNVARCHARandVARBINARY, including a converter invocation count assertion to detect the previously-masked behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
mssql_python/cursor.py | Restricts the legacy SQL_WVARCHAR fallback converter to string/binary-mapped columns during converter map construction. |
tests/test_003_connection.py | Adds a regression test ensuring SQL_WVARCHAR converters do not act as a catch-all for non-string columns and asserts call count. |
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changes
Summary
📋 Files Needing Attention📉 Files with overall lowest coverage (click to expand)mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 59.9%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 76.3%
mssql_python.__init__.py: 77.3%
mssql_python.row.py: 77.6%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 83.7%
mssql_python.connection.py: 84.7%🔗 Quick Links
|
Uh oh!
There was an error while loading. Please reload this page.
Address review feedback on PR #692 by covering four additional scenarios in tests/test_003_connection.py, all live-validated: - NULL string: a NULL NVARCHAR stays None and the WVARCHAR converter is never invoked (spy asserts zero calls). - VARCHAR column: the gated WVARCHAR fallback still applies to VARCHAR (mapped type str), receiving the value as UTF-16LE bytes. - Mixed result set (INT + NVARCHAR + DECIMAL): the converter fires only on the string column exactly once; INT/DECIMAL are untouched. - Cached converter map across multiple fetches: a multi-row fetchall converts every row consistently and never touches the INT column.
Uh oh!
There was an error while loading. Please reload this page.
…ar-catchall # Conflicts: # mssql_python/cursor.py # tests/test_003_connection.py
…ng (GH #691) Adds test_output_converter_int_column_converter_vs_wvarchar_gating_gh691 per reviewer request. Positive: a converter registered for the INT column (SQL_INTEGER code) still fires and receives the raw integer. Negative: with only a SQL_WVARCHAR converter registered, it must not sneak onto the INT column -- value stays 42 and the spy converter is never invoked (call count asserted, robust against the exception-swallowing apply path).
Uh oh!
There was an error while loading. Please reload this page.
### Work Item / Issue Reference > [AB#47087](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/47087) ------------------------------------------------------------------- ### Summary Release mssql-python v1.13.0. Version bump to 1.13.0. Updates `mssql_python/__init__.py`, `setup.py`, `PyPI_Description.md`, and the README "Important Note" section. Bundled `mssql_py_core` bumped from 0.1.7 to 0.1.8 (no source/API changes — dev-nightly to stable pin). #### Enhancements - **ODBC driver ships exclusively via `mssql-python-odbc` (Phase 2)** — The `libs/` fallback introduced in v1.12.0 has been removed. `mssql-python` now hard-depends on `mssql-python-odbc==18.6.2.1`; `pip install mssql-python` still pulls the driver package transparently. Smaller wheels; driver binaries managed independently (#693). - **Apache Arrow bulk copy** — New `Cursor.bulkcopy_arrow(table_name, source)` method for high-performance bulk loading from `pyarrow.Table` / `RecordBatch` / Arrow C Data Interface sources; classic `bulkcopy()` now raises `TypeError` for Arrow inputs and steers users to the new method (#665). - **`token_provider=` parameter for Azure Identity credentials** — `connect()` accepts any credential with a `.get_token(scope)` method (`DefaultAzureCredential`, `AzureCliCredential`, `ManagedIdentityCredential`, …). Mutually exclusive with `Authentication=` in the connection string (#603, issue #577). - **Identity-aware connection pooling with token-expiry refresh** — Pool now keys on security context, preventing cross-identity connection leaks; token acquisition deferred to pool-misses; connections with near-expiry tokens refreshed automatically (#660, issues #651, #659). #### Bug Fixes - **Silent zero-row `executemany` batches on late NULLs** — Fixed numeric array parameter binding paths (`TINYINT`/`SMALLINT`/`INT`/`FLOAT`) that left indicator slots uninitialized when a NULL appeared partway through the batch (#702, issue #670). - **`SQL_WVARCHAR` output converter applied as catch-all to non-string columns** — Fallback now gated on `str`/`bytes` mapped types (#692, issue #691). - **Integer-keyed output converters silently never fired** — `add_output_converter(SQL_DECIMAL, ...)` and other integer SQL type code keys now dispatch correctly (pyodbc parity) (#690, issue #684). - **`RecordBatchReader.Close()` for Arrow result sets** — `Cursor.arrow_reader()` now returns a wrapper whose `.close()` releases server-side resources and leaves the parent cursor usable (#644, issue #643). - **`AttributeError` in `Cursor.__del__` on partially-initialized cursor** — `__init__` sets `closed`/`hstmt` before any raise; `__del__` uses correct `sys.is_finalizing()` guard (#646, issue #642). #### Version Bump - `mssql_python/__init__.py`: `__version__ = "1.13.0"` - `setup.py`: `version="1.13.0"` - `PyPI_Description.md`: `## What's new in v1.13.0` section refreshed - `README.md`: "Important Note" updated for Phase 2 (no more `libs/` fallback, `mssql-python-odbc==18.6.2.1`)
Work Item / Issue Reference
Summary
A SQL_WVARCHAR output converter was being applied as an unconditional
catch-all to non-string columns.
Cursor._build_converter_mapfell back to theconverter registered for
SQL_WVARCHARfor any column that had no directtype-keyed converter, so registering a single
SQL_WVARCHARconverter mangledINT/DECIMAL/DATEvalues (e.g.42came back converted).This PR gates that legacy fallback on the column's mapped Python type being
str/bytes, mirroring theisinstance(value, (str, bytes))guard alreadypresent in
Row._apply_output_converters. The optimized and fallback applypaths now behave consistently.
Root cause
sql_type = desc[1]is the column's mapped Python type. The fallback ranwhenever no direct converter matched, regardless of the column type:
Row._apply_output_convertersgates the same fallback onisinstance(value, (str, bytes)), so the two paths disagreed.Fix
bytesis intentionally included for parity withRow._apply_output_converters(a
SQL_WVARCHARconverter still applies toVARBINARY, which the new testasserts). Also removed the now-redundant in-loop
ConstantsDDBCimport.Test
tests/test_003_connection.py::test_output_converter_wvarchar_not_catchall_for_non_string_columns_gh691registers only a
SQL_WVARCHARconverter and selectsINT,DECIMAL,DATE,NVARCHAR, andVARBINARYcolumns. It asserts the non-string columns areuntouched, the string/binary columns are converted, and — via a spy — that the
converter is invoked exactly twice (only the
str/bytescolumns).The call-count assertion matters: the optimized apply path swallows converter
exceptions, so a plain
value.decode()converter raisesAttributeErroronint/Decimal/dateand is silently swallowed. A value-only assertion wouldtherefore pass with and without the fix. Verified (via
git stash) that thetest fails on the unfixed code and passes once the fallback is gated.
Validation
black --check --line-length=100clean on both files.tests/test_003_connection.py: 151 passed, 2 skipped.tests/test_004_cursor.py: 517 passed, 4 skipped.