Skip to content

FIX: SQLDescribeParam ordinal remapping for VARBINARY/BINARY NULL - #654

Merged
Jahnvi Thakkar (jahnvi480) merged 11 commits into
mainfrom
jahnvi/fix-sqldescribeparam-varbinary-627
Jul 6, 2026
Merged

FIX: SQLDescribeParam ordinal remapping for VARBINARY/BINARY NULL#654
Jahnvi Thakkar (jahnvi480) merged 11 commits into
mainfrom
jahnvi/fix-sqldescribeparam-varbinary-627

Conversation

@jahnvi480

@jahnvi480Jahnvi Thakkar (jahnvi480) commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Work Item / Issue Reference

AB#45521

GitHub Issue: #627


Summary

This pull request significantly improves how the driver resolves SQL types for NULL parameters, especially when inserting into temp tables or table variables. It proactively resolves unknown NULL parameter types before binding, emits actionable Python warnings when automatic type resolution fails, and provides clear guidance on using setinputsizes to avoid SQL Server implicit conversion errors. Additionally, the code is refactored for clarity and efficiency, and minor formatting improvements are made.

Improvements to NULL parameter type resolution:

  • Added a new PreResolveUnknownNullTypes function to proactively resolve unknown NULL SQL types for both single and batch execution paths, ensuring all types are determined before any parameters are bound. This prevents driver errors with binary NULLs and improves reliability.
  • When SQL type resolution falls back to SQL_VARCHAR (because SQLDescribeParam fails), the code now emits a Python warning with detailed guidance on how to use cursor.setinputsizes() to specify the correct type, preventing common SQL Server errors with BINARY/VARBINARY columns.

Documentation and user guidance:

  • Expanded the setinputsizes docstring in cursor.py to document the temp table/NULL issue and provide a concrete usage example for binary columns.

Code structure and clarity:

  • Refactored parameter binding code to avoid redundant type resolution during binding, since types are now resolved in advance.
  • Added a bool isFallback field to DescribedParamInfo to track when a fallback type is used.

Minor improvements:

  • Made minor formatting and whitespace changes for improved readability.

)
Root cause: When SQLDescribeParam is called AFTER some parameters are already
bound via SQLBindParameter, the ODBC Driver Manager remaps parameter ordinals,
causing SQLSTATE 07009 errors or returning wrong types for VARBINARY/BINARY
columns.
Fix: Pre-resolve ALL unknown NULL parameter types via SQLDescribeParam BEFORE
any SQLBindParameter call, using the existing per-handle cache. This ensures
stable ordinals since no parameters are bound at describe time.
Changes:
- ddbc_bindings.cpp: Add PreResolveUnknownNullTypes() that scans paramInfos
for SQL_C_DEFAULT + SQL_UNKNOWN_TYPE and resolves them via cache before
binding. Called from both BindParameters (execute) and BindParameterArray
(executemany) paths.
- ddbc_bindings.cpp: Release GIL during SQLDescribeParam (network call to
sp_describe_undeclared_parameters).
- ddbc_bindings.cpp: Cache both successful and fallback results to avoid
repeated network calls on statement reuse. Emit UserWarning with
setinputsizes guidance when SQLDescribeParam fails.
- ddbc_bindings.h: Add isFallback field to DescribedParamInfo to distinguish
successful describes from fallback values.
- cursor.py: Document setinputsizes workaround for temp table BINARY columns
in docstring.
- tests: Add 7 regression tests covering NULL VARBINARY/BINARY with non-NULL
params, executemany, statement reuse, temp table warning, and setinputsizes
workaround.
Performance: Zero overhead for common case (no NULL params) due to early-exit
scan. Cached path adds <1us. Fallback caching gives 2.5x speedup on repeated
temp table queries vs prior approach.
Closes#627
CopilotAI review requested due to automatic review settings June 30, 2026 09:31
@github-actionsgithub-actionsBot added the pr-size: medium Moderate update size label Jun 30, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses GH-627 by preventing SQLDescribeParam ordinal remapping failures when NULL parameters (notably BINARY/VARBINARY) are described after earlier parameters have already been bound. It does this by pre-resolving unknown NULL parameter SQL types before any binding occurs, adds an end-user warning when SQLDescribeParam fails and the driver must fall back to SQL_VARCHAR, and adds regression coverage and documentation.

Changes:

  • Add a pre-scan (PreResolveUnknownNullTypes) that resolves unknown NULL parameter SQL types ahead of binding for both execute() and executemany() paths.
  • Emit a Python UserWarning when SQLDescribeParam fails (fallback to SQL_VARCHAR) with guidance to use setinputsizes.
  • Add GH-627 regression tests and expand setinputsizes documentation with guidance for temp table/table variable NULL binary cases.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

FileDescription
mssql_python/pybind/ddbc_bindings.cppPre-resolves unknown NULL types before binding; adds SQLDescribeParam fallback warning and refactors binding to avoid interleaved describe+bind.
mssql_python/pybind/ddbc_bindings.hExtends DescribedParamInfo with isFallback metadata.
mssql_python/cursor.pyUpdates setinputsizes docstring with GH-627 guidance and example usage.
tests/test_004_cursor.pyAdds regression tests covering execute/executemany ordering, reuse path, and temp-table warning/workaround behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadmssql_python/pybind/ddbc_bindings.cpp Outdated
Comment threadmssql_python/cursor.py Outdated
Comment threadtests/test_004_cursor.py Outdated
Comment threadtests/test_004_cursor.py
Comment threadtests/test_004_cursor.py Outdated
- Warning: clarify 0-based index, show example with all params typed
- Docstring: remove incorrect None placeholder claim, use ConstantsDDBC
- Tests: drop temp tables after use, use named constants instead of magic numbers
@github-actions

github-actionsBot commented Jun 30, 2026

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

87%


🎯 Overall Coverage

80%


📈 Total Lines Covered:6738 out of 8326
📁 Project:mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

  • mssql_python/pybind/ddbc_bindings.cpp (87.9%): Missing lines 542,552-553,731

Summary

  • Total: 33 lines
  • Missing: 4 lines
  • Coverage: 87%

mssql_python/pybind/ddbc_bindings.cpp

Lines 538-546

538 }
539540// GH-627: Resolve unknown NULL SQL types before any SQLBindParameter calls.541// Some drivers remap parameter ordinals during describe when parameters have
! 542// already been bound, so interleaving describe+bind can fail for binary NULLs.543// When `params` is provided (execute path), an additional py::none check is544// performed; for executemany (array path), SQL_C_DEFAULT already guarantees545// all values in that column are NULL, so no Python-level check is needed.546staticvoidPreResolveUnknownNullTypes(SqlHandle& handle, SQLHANDLE hStmt,

Lines 548-557

548const py::list* params = nullptr) {
549if (paramInfos.empty())
550return;
551 ! 552for (size_t paramIndex = 0; paramIndex < paramInfos.size(); ++paramIndex) {
! 553 ParamInfo& paramInfo = paramInfos[paramIndex];
554if (paramInfo.paramCType != SQL_C_DEFAULT || paramInfo.paramSQLType != SQL_UNKNOWN_TYPE) {
555continue;
556 }
557// For execute(), verify the actual value is None (mixed columns possible).

Lines 727-735

727caseSQL_C_DEFAULT: {
728if (!py::isinstance<py::none>(param)) {
729ThrowStdException(MakeParamMismatchErrorStr(paramInfo.paramCType, paramIndex));
730 }
! 731 dataPtr = nullptr; // GH-627: type resolved by PreResolveUnknownNullTypes.732 strLenOrIndPtr = AllocateParamBuffer<SQLLEN>(paramBuffers);
733 *strLenOrIndPtr = SQL_NULL_DATA;
734 bufferLength = 0;
735break;


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 59.9%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection.cpp: 76.2%
mssql_python.pybind.ddbc_bindings.cpp: 76.2%
mssql_python.__init__.py: 77.3%
mssql_python.row.py: 77.6%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.logging.py: 85.5%
mssql_python.connection.py: 85.6%

🔗 Quick Links

⚙️ Build Summary📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@jahnvi480
Jahnvi Thakkar (jahnvi480) marked this pull request as draft June 30, 2026 13:02
@jahnvi480
Jahnvi Thakkar (jahnvi480) marked this pull request as ready for review July 1, 2026 08:44
gargsaumya
gargsaumya previously approved these changes Jul 3, 2026
Comment threadmssql_python/pybind/ddbc_bindings.cpp Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core fix is right, and solves the issue
comments mostly are all on the warning introduced and impl around it which can cause future problems, requesting changes

Comment threadmssql_python/pybind/ddbc_bindings.cpp Outdated
Comment threadmssql_python/pybind/ddbc_bindings.cpp Outdated
Comment threadmssql_python/pybind/ddbc_bindings.cpp Outdated
Comment threadmssql_python/pybind/ddbc_bindings.cpp Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jahnvi480
Jahnvi Thakkar (jahnvi480) merged commit 98bb35a into mainJul 6, 2026
29 checks passed
@gargsaumyagargsaumya mentioned this pull request Jul 10, 2026
Gaurav Sharma (bewithgaurav) pushed a commit that referenced this pull request Jul 10, 2026
Release mssql-python v1.11.0.
[AB#46332](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/46332)
### Summary
Version bump to 1.11.0. Updates `mssql_python/__init__.py`, `setup.py`,
and `PyPI_Description.md`. Bundled `mssql_py_core` bumped from 0.1.5 to
0.1.6.
#### Bug Fixes
- **SSH-tunnel / in-process forwarder deadlock** — Release the GIL
around blocking ODBC round-trips in the teardown path and
`SQLDescribeParam`, so `close()` and parametrized queries with `None`
values no longer deadlock in-process TCP forwarder setups (#604, issue
#565).
- **BINARY/VARBINARY NULL parameters in temp tables and table
variables** — Proactively resolve unknown NULL parameter types before
binding and emit actionable `setinputsizes()` guidance on `SQL_VARCHAR`
fallback (#654, issue #627).
- **Context manager transaction semantics** — `Connection.__exit__` now
commits on clean exit and rolls back on exception when
`autocommit=False`, instead of always rolling back (#639, issue #635).
- **macOS Apple Silicon import failure** — `configure_dylibs.sh`
rewrites bundled ODBC dylib dependencies to `@loader_path` for every
architecture in the universal2 wheel, fixing `import mssql_python` on
clean Apple Silicon machines (#661, issue #656).
- **Service Principal bulk copy freeze** — Fixed a GIL-deadlock in the
Rust core that froze bulk copy under `ActiveDirectoryServicePrincipal`
auth; picked up via the `mssql_py_core` 0.1.6 bump (#666, issue #662).
#### Version Bump
- `mssql_python/__init__.py`: `__version__ = "1.11.0"`
- `setup.py`: `version="1.11.0"`
- `PyPI_Description.md`: `## What's new in v1.11.0` section refreshed.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: mediumModerate update size

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@jahnvi480@bewithgaurav@gargsaumya