FIX: Redact parameter values from executemany Decimal-conversion error

[Sumit Sarabhai (sumitmsft)]

Work Item / Issue Reference

AB#47300

---

Summary

The executemany() Decimal/NUMERIC conversion path embedded the entire parameter row into the raised ValueError:

ValueError(f"Failed to convert parameter at row {row}, column {i} to Decimal: {e}")

Here row is the full parameter tuple, so every column value in a failing row (potentially PII such as SSNs, emails, names, or balances) was placed into the exception message. That message propagates to caller error handlers, tracebacks, and APM/log shippers (Sentry, Splunk, App Insights) even when driver DEBUG logging is never enabled, because uncaught exceptions surface by default.

This change makes the error metadata-only: it reports the row index, column index, and the value's type name, and never the value or the row.

Before

Failed to convert parameter at row ('Jane Doe', '123-45-6789', 'jane@x.com', 'N/A'), column 3 to Decimal: [<class 'decimal.ConversionSyntax'>]

After

Failed to convert parameter to Decimal at row 0, column 3 (value type: str)

Hardening: value-bearing exception cause

Redacting only the outer message is not enough. raise ... from e chains the original exception, which surfaces through __cause__ and traceback.format_exc() -- exactly what APM/log shippers capture. For an ordinary bad string the chained cause is the value-free decimal.InvalidOperation (ConversionSyntax), but for a value whose str() raises with content (or any error echoing the input) the sensitive text would still leak via the cause.

The conversion now splits str(val) from the decimal parse and chains onlydecimal.DecimalException, which is proven value-free (it never echoes the input). str(val) failures and any other unexpected error are re-raised with the chain suppressed (from None), so the metadata-only guarantee holds across __cause__ and formatted tracebacks, not just str(exc).

Changes

• mssql_python/cursor.py: enumerate rows for an index; raise a value-free, metadata-only ValueError; chain only the value-free decimal.DecimalException and suppress the cause (from None) for str(val) / unexpected failures.
• tests/test_004_cursor.py: strengthen the unconvertible-value test to assert the sensitive value and raw row are absent from both the message and the fully formatted traceback; add test_setinputsizes_sql_decimal_str_raises_no_leak covering a parameter whose str() raises a secret (asserts __cause__ is None and the secret is absent from the traceback).

[Copilot]

Pull request overview

This PR updates the executemany() DECIMAL/NUMERIC conversion error path to avoid leaking full parameter rows (potentially containing PII) into exception messages, while preserving debugging detail via exception chaining.

Changes:

• Redacts parameter values/rows from the ValueError raised during DECIMAL conversion in Cursor.executemany(), replacing them with row/column index + value type metadata.
• Strengthens the integration test to assert that sensitive values and raw parameter-row representations are not present in the raised error message.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
mssql_python/cursor.py	Changes DECIMAL/NUMERIC conversion failure message to include only row/column indices and value type (no value/row), using exception chaining.
tests/test_004_cursor.py	Adds assertions that the error message includes metadata and does not include the sensitive value / raw row.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 81%

🎯 Overall Coverage 82%

📈 Total Lines Covered:7373 out of 8970 📁 Project:mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/cursor.py (81.8%): Missing lines 2696-2697

Summary

• Total: 11 lines
• Missing: 2 lines
• Coverage: 81%

mssql_python/cursor.py

Lines 2692-2701

2692try:
2693processed_row[i] =format(decimal.Decimal(val_text), "f")
2694exceptdecimal.DecimalExceptionase:
2695raiseValueError(err_msg) frome
! 2696exceptException: # pylint: disable=broad-exception-caught
! 2697raiseValueError(err_msg) fromNone2698processed_parameters.append(processed_row)
26992700# Now transpose the processed parameters2701columnwise_params, row_count=self._transpose_rowwise_to_columnwise(processed_parameters)

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 59.9%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 76.6%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 84.3%
mssql_python.logging.py: 85.5%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[gargsaumya]

This block also still logs the first 5 full parameter rows ([seq_of_parameters[:5]])- the same PII this PR redacts from the exception. Redact or gate this too

[Gaurav Sharma (bewithgaurav)]

+1

[Gaurav Sharma (bewithgaurav)]

no blockers as such, added a test suggestion - requesting changes

[Gaurav Sharma (bewithgaurav)]

test suggestion: this is an imp branch, and adding tests would be highly beneficial here
the coverage bot flags 2696-2697 as the only miss on the diff

asking since it is reachable cheaply and deterministically.

format(Decimal("1e999999999999999999"), "f") raises MemoryError, not a DecimalException, in 0.1ms. the fixed-point expansion length is rejected up front so nothing is allocated.

suggesting adding the below - will take the diff from 81% to 100%):

deftest_setinputsizes_sql_decimal_non_decimal_error_no_leak(db_connection):
"""A non-DecimalException during conversion must not chain a cause (GH-503)."""cursor=db_connection.cursor()
huge_exponent="1e999999999999999999"cursor.execute("DROP TABLE IF EXISTS #test_sis_dec_nondec")
try:
cursor.execute("CREATE TABLE #test_sis_dec_nondec (Price DECIMAL(18,2))")
cursor.setinputsizes([(mssql_python.SQL_DECIMAL, 18, 2)])
withpytest.raises(ValueError) asexc_info:
cursor.executemany(
"INSERT INTO #test_sis_dec_nondec (Price) VALUES (?)",
[(huge_exponent,)],
)
message=str(exc_info.value)
assert"Failed to convert parameter"inmessageassert"row 0"inmessageassert"column 0"inmessageasserthuge_exponentnotinmessageassertexc_info.value.__cause__isNoneformatted="".join(
traceback.format_exception(
type(exc_info.value), exc_info.value, exc_info.value.__traceback__
)
)
asserthuge_exponentnotinformattedfinally:
cursor.execute("DROP TABLE IF EXISTS #test_sis_dec_nondec")