FEAT: Add table-valued parameter support

[Gaurav Sharma (bewithgaurav)]

Work Item / Issue Reference

ADO Work Item: Fixed AB#48177

---

Summary

Adds input table-valued parameter support for stored procedure calls using materialized lists or tuples of rows. Prepared ODBC metadata confirms TVP destinations and resolves their declared schemas, allowing empty inputs and NULL values without adding a public wrapper. The implementation also covers multiple TVPs and statement reuse, validates malformed rows, and explicitly defers bounded-memory streaming and the Rust ODBC provider.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 88%

🎯 Overall Coverage 84%

📈 Total Lines Covered: 8900 out of 10565 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/cursor.py (100%)
• mssql_python/pybind/ddbc_bindings.cpp (87.7%): Missing lines 511,576,579,597-598,600-602,627-628,661-662,670-671,682-683,695-696,729-730,754-755,765-766,783-784,786-787,812-813,881-882,888-889,894-895,905-906,913-914,916-918,956-964,2913,2917,2926
• mssql_python/pybind/ddbc_bindings.h (100%)
• mssql_python/pybind/param_detect.hpp (100%)

Summary

• Total: 499 lines
• Missing: 55 lines
• Coverage: 88%

mssql_python/pybind/ddbc_bindings.cpp

Lines 507-515

  507         if (currentInteger) {
  508             continue;
  509         } else if (currentText) {
  510             if (current.paramSQLType == SQL_WVARCHAR) {
! 511                 merged.paramSQLType = SQL_WVARCHAR;
  512             }
  513             SQLULEN size =
  514                 current.isDAE ? static_cast<SQLULEN>(current.utf16Len) : current.columnSize;
  515             maxSize = std::max(maxSize, size);

---

Lines 572-583

  572             // Preserve each coefficient's scale using exact text, while binding a common
  573             // source NUMERIC precision/scale. Buffer capacity is not numeric precision.
  574             py::str formatSpec("f");
  575             info.paramCType = SQL_C_CHAR;
! 576             info.bufferSize = 1;
  577             for (size_t rowIndex = 0; rowIndex < values.size(); ++rowIndex) {
  578                 if (values[rowIndex].is_none()) {
! 579                     continue;
  580                 }
  581                 py::object formatted =
  582                     steal(PyObject_Format(values[rowIndex].ptr(), formatSpec.ptr()));
  583                 if (!formatted)

---

Lines 593-606

  593         columnValues[columnIndex] = std::move(detectedValues);
  594     }
  595     return columnInfos;
  596 }
! 597 
! 598 static SQLRETURN ReadDescriptorString(SQLHDESC descriptor, SQLSMALLINT recordNumber,
  599                                       SQLSMALLINT fieldIdentifier, std::u16string& value) {
! 600     SQLWCHAR buffer[256] = {};
! 601     SQLINTEGER length = 0;
! 602     SQLRETURN rc = SQLGetDescField_ptr(descriptor, recordNumber, fieldIdentifier, buffer,
  603                                        static_cast<SQLINTEGER>(sizeof(buffer)), &length);
  604     if (!SQL_SUCCEEDED(rc)) {
  605         return rc;
  606     }

---

Lines 623-632

  623     throw std::runtime_error(operation);
  624 }
  625 
  626 class TableMetadataScope {
! 627   public:
! 628     explicit TableMetadataScope(SQLHANDLE statementHandle) : handle(statementHandle) {}
  629     ~TableMetadataScope() {
  630         // Error paths can leave unread metadata; closing it can require network I/O.
  631         if (!is_python_finalizing() && PyGILState_Check()) {
  632             py::gil_scoped_release release;

---

Lines 657-666

  657 
  658     SQLHANDLE hMetadataStmt = metadataHandle->get();
  659     {
  660         py::gil_scoped_release release;
! 661         SQLFreeStmt_ptr(hMetadataStmt, SQL_CLOSE);
! 662         SQLFreeStmt_ptr(hMetadataStmt, SQL_RESET_PARAMS);
  663     }
  664 
  665     SQLRETURN rc = SQLSetStmtAttr_ptr(hMetadataStmt, SQL_ATTR_METADATA_ID,
  666                                       reinterpret_cast<SQLPOINTER>(static_cast<intptr_t>(SQL_TRUE)),

---

Lines 666-675

  666                                       reinterpret_cast<SQLPOINTER>(static_cast<intptr_t>(SQL_TRUE)),
  667                                       SQL_IS_INTEGER);
  668     if (!SQL_SUCCEEDED(rc)) {
  669         ThrowTableError(metadataHandle, rc, "Failed to enable metadata identifier mode");
! 670     }
! 671     TableMetadataScope metadataScope(hMetadataStmt);
  672 
  673     rc = SQLSetStmtAttr_ptr(
  674         hMetadataStmt, SQL_SOPT_SS_NAME_SCOPE,
  675         reinterpret_cast<SQLPOINTER>(static_cast<intptr_t>(SQL_SS_NAME_SCOPE_TABLE_TYPE)),

---

Lines 678-687

  678         ThrowTableError(metadataHandle, rc, "Failed to select table-type metadata scope");
  679     }
  680     {
  681         py::gil_scoped_release release;
! 682         rc = SQLColumns_ptr(hMetadataStmt,
! 683                             catalog.empty() ? nullptr : reinterpretU16stringAsSqlWChar(catalog),
  684                             catalog.empty() ? 0 : SQL_NTS,
  685                             schema.empty() ? nullptr : reinterpretU16stringAsSqlWChar(schema),
  686                             schema.empty() ? 0 : SQL_NTS, reinterpretU16stringAsSqlWChar(typeName),
  687                             SQL_NTS, nullptr, 0);

---

Lines 691-700

  691     }
  692 
  693     std::vector<DescribedParamInfo> columns;
  694     while (true) {
! 695         {
! 696             py::gil_scoped_release release;
  697             rc = SQLFetch_ptr(hMetadataStmt);
  698         }
  699         if (rc == SQL_NO_DATA) {
  700             break;

---

Lines 725-734

  725             }
  726         }
  727         if (!SQL_SUCCEEDED(rc)) {
  728             ThrowTableError(metadataHandle, rc, "Failed to read TVP column metadata");
! 729         }
! 730 
  731         columns.push_back({
  732             sqlType, static_cast<SQLULEN>(std::max<SQLINTEGER>(columnSize, 0)), decimalDigits,
  733         });
  734     }

---

Lines 750-759

  750 
  751         if (handle.tvpCache.find(static_cast<int>(paramIndex)) == handle.tvpCache.end()) {
  752             DescribedParamInfo described;
  753             SQLSMALLINT nullable;
! 754             RETCODE rc;
! 755             {
  756                 py::gil_scoped_release release;
  757                 rc = SQLDescribeParam_ptr(hStmt, static_cast<SQLUSMALLINT>(paramIndex + 1),
  758                                           &described.sqlType, &described.columnSize,
  759                                           &described.decimalDigits, &nullable);

---

Lines 761-770

  761             if (!SQL_SUCCEEDED(rc)) {
  762                 return rc;
  763             }
  764             if (described.sqlType != SQL_SS_TABLE) {
! 765                 throw py::type_error(
! 766                     "Nested row sequences can only be bound to a table-valued parameter");
  767             }
  768 
  769             SQLHDESC implementationDescriptor = nullptr;
  770             rc = SQLGetStmtAttr_ptr(hStmt, SQL_ATTR_IMP_PARAM_DESC, &implementationDescriptor, 0,

---

Lines 779-791

  779             const SQLSMALLINT recordNumber = static_cast<SQLSMALLINT>(paramIndex + 1);
  780             rc = ReadDescriptorString(implementationDescriptor, recordNumber,
  781                                       SQL_CA_SS_CATALOG_NAME, catalog);
  782             if (SQL_SUCCEEDED(rc)) {
! 783                 rc = ReadDescriptorString(implementationDescriptor, recordNumber,
! 784                                           SQL_CA_SS_SCHEMA_NAME, schema);
  785             }
! 786             if (SQL_SUCCEEDED(rc)) {
! 787                 rc = ReadDescriptorString(implementationDescriptor, recordNumber,
  788                                           SQL_CA_SS_TYPE_NAME, typeName);
  789             }
  790             if (!SQL_SUCCEEDED(rc)) {
  791                 return rc;

---

Lines 808-817

  808                                          int paramIndex, const py::handle& param,
  809                                          std::vector<std::shared_ptr<void>>& paramBuffers,
  810                                          const std::string& charEncoding) {
  811     SqlHandle& handle = *statementHandle;
! 812     const py::sequence rows = py::reinterpret_borrow<py::sequence>(param);
! 813     const SQLSMALLINT recordNumber = static_cast<SQLSMALLINT>(paramIndex + 1);
  814 
  815     size_t columnCount = 0;
  816     py::list columnValues;
  817     std::vector<ParamInfo> columnInfos;

---

Lines 877-886

  877 
  878     auto* typeName = AllocateParamBuffer<std::u16string>(paramBuffers, tableMetadata.typeName);
  879     auto* schema = AllocateParamBuffer<std::u16string>(paramBuffers, tableMetadata.schema);
  880     auto* rowCount = AllocateParamBuffer<SQLLEN>(paramBuffers);
! 881     // TVPs overload ColumnSize with row capacity and the indicator with the rows available.
! 882     *rowCount = rows.empty() ? SQL_DEFAULT_PARAM : static_cast<SQLLEN>(rows.size());
  883 
  884     SQLRETURN rc = SQLBindParameter_ptr(
  885         hStmt, static_cast<SQLUSMALLINT>(recordNumber), SQL_PARAM_INPUT, SQL_C_BINARY, SQL_SS_TABLE,
  886         static_cast<SQLULEN>(rows.size()), 0, typeName->data(),

---

Lines 884-899

  884     SQLRETURN rc = SQLBindParameter_ptr(
  885         hStmt, static_cast<SQLUSMALLINT>(recordNumber), SQL_PARAM_INPUT, SQL_C_BINARY, SQL_SS_TABLE,
  886         static_cast<SQLULEN>(rows.size()), 0, typeName->data(),
  887         static_cast<SQLLEN>(typeName->size() * sizeof(SQLWCHAR)), rowCount);
! 888     if (!SQL_SUCCEEDED(rc)) {
! 889         return rc;
  890     }
  891     if (!schema->empty()) {
  892         SQLHDESC implementationDescriptor = nullptr;
  893         rc = SQLGetStmtAttr_ptr(hStmt, SQL_ATTR_IMP_PARAM_DESC, &implementationDescriptor, 0,
! 894                                 nullptr);
! 895         if (!SQL_SUCCEEDED(rc)) {
  896             return rc;
  897         }
  898         rc = SQLSetDescField_ptr(implementationDescriptor, recordNumber, SQL_CA_SS_SCHEMA_NAME,
  899                                  reinterpretU16stringAsSqlWChar(*schema),

---

Lines 901-910

  901         if (!SQL_SUCCEEDED(rc)) {
  902             return rc;
  903         }
  904     }
! 905     if (rows.empty()) {
! 906         return rc;
  907     }
  908 
  909     rc = SQLSetStmtAttr_ptr(hStmt, SQL_SOPT_SS_PARAM_FOCUS,
  910                             reinterpret_cast<SQLPOINTER>(static_cast<intptr_t>(recordNumber)),

---

Lines 909-922

  909     rc = SQLSetStmtAttr_ptr(hStmt, SQL_SOPT_SS_PARAM_FOCUS,
  910                             reinterpret_cast<SQLPOINTER>(static_cast<intptr_t>(recordNumber)),
  911                             SQL_IS_INTEGER);
  912     if (!SQL_SUCCEEDED(rc)) {
! 913         return rc;
! 914     }
  915 
! 916     try {
! 917         rc = BindParameterArray(handle, hStmt, columnValues, columnInfos, rows.size(),
! 918                                 paramBuffers, charEncoding);
  919         if (!SQL_SUCCEEDED(rc)) {
  920             // Successful focus restoration clears the original statement diagnostic.
  921             ThrowTableError(statementHandle, rc, "Failed to bind TVP columns");
  922         }

---

Lines 952-968

  952         void* dataPtr = nullptr;
  953         SQLLEN bufferLength = 0;
  954         SQLLEN* strLenOrIndPtr = nullptr;
  955 
! 956         if (paramInfo.isTVP) {
! 957             RETCODE rc = BindTableValuedParameter(statementHandle, hStmt, paramIndex, param,
! 958                                                  paramBuffers, charEncoding);
! 959             if (!SQL_SUCCEEDED(rc)) {
! 960                 return rc;
! 961             }
! 962             continue;
! 963         }
! 964 
  965         // TODO: Add more data types like money, guid, interval, TVPs etc.
  966         switch (paramInfo.paramCType) {
  967             case SQL_C_CHAR: {
  968                 if (!py::isinstance<py::str>(param) && !py::isinstance<py::bytearray>(param) &&

---

Lines 2909-2921

  2909                             if (encodedStr.size() > valueBufferSize) {
  2910                                 LOG("BindParameterArray: String/binary too "
  2911                                     "long - param_index=%d, row=%zu, size=%zu, "
  2912                                     "max=%zu",
! 2913                                     paramIndex, i, encodedStr.size(), valueBufferSize);
  2914                                 ThrowStdException("Input exceeds column size at index " +
  2915                                                   std::to_string(i));
  2916                             }
! 2917                             std::memcpy(charArray + i * (valueBufferSize + 1), encodedStr.c_str(),
  2918                                         encodedStr.size());
  2919                             strLenOrIndArray[i] = static_cast<SQLLEN>(encodedStr.size());
  2920                         }
  2921                     }

---

Lines 2922-2930

  2922                     LOG("BindParameterArray: SQL_C_CHAR/BINARY bound - "
  2923                         "param_index=%d",
  2924                         paramIndex);
  2925                     dataPtr = charArray;
! 2926                     bufferLength = valueBufferSize + 1;
  2927                     break;
  2928                 }
  2929                 case SQL_C_BIT: {
  2930                     LOG("BindParameterArray: Binding SQL_C_BIT array - "

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 63.2%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.row.py: 77.6%
mssql_python.pybind.ddbc_bindings.cpp: 79.4%
mssql_python.pybind.connection.connection_pool.cpp: 82.4%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.py_type_cache.hpp: 91.6%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

🟡 Changes recommended

The TVP execution path has critical single-TVP argument handling and moderate metadata-timeout diagnostic defects.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds ODBC-backed table-valued parameter support for stored procedure execution using materialized rows.

Changes:

• Adds TVP detection, metadata resolution, validation, and row binding.
• Supports empty, NULL, typed, multiple, and reusable TVPs.
• Adds integration/regression tests and changelog documentation.

File summaries

File	Summary
tests/test_028_table_valued_parameters.py	Adds comprehensive TVP integration coverage.
tests/test_004_cursor.py	Adds embedded-NUL regression coverage.
mssql_python/pybind/param_detect.hpp	Adds TVP parameter detection and binding metadata.
mssql_python/pybind/ddbc_bindings.h	Defines TVP metadata structures and ODBC declarations.
mssql_python/pybind/ddbc_bindings.cpp	Resolves TVP schemas and binds rows through ODBC.
mssql_python/cursor.py	Manages TVP execution and provider handling; contains critical single-TVP normalization and moderate timeout-diagnostic issues.
CHANGELOG.md	Documents TVP support and provider limitations.

Review details

Suppressed comments (1)

mssql_python/cursor.py:1014

• When statement_handle is supplied for the TVP metadata HSTMT, the timeout failure check still passes self.hstmt, so diagnostics are read from the wrong statement and a failed metadata timeout can be silently ignored. Pass the selected handle to check_error as well.

                    statement_handle or self.hstmt,
                    ddbc_sql_const.SQL_ATTR_QUERY_TIMEOUT.value,
                    timeout_value,

• Files reviewed: 7/7 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Unresolved critical and moderate issues remain in TVP binding and cursor handling.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

mssql_python/cursor.py:1012

• When an alternate handle is supplied for the TVP metadata statement, the timeout is set on that handle but check_error still reads diagnostics from self.hstmt. If setting the metadata handle attribute fails, the wrong statement's diagnostics are inspected and the exception is swallowed by this try/except, so metadata discovery can run without the requested query timeout. Check the same target handle that was passed to DDBCSQLSetStmtAttr.

mssql_python/cursor.py:1740

• The documented single-TVP call does not work for an empty row sequence. With execute("EXEC proc ?", ([],)) (or ((),)), the existing length-one unwrapping sets actual_params to the empty sequence and then converts it to parameters=[]; the new TVP path is never reached and SQL receives zero bound parameters. Preserve the outer parameter tuple for empty list/tuple TVPs, or add equivalent disambiguation, so an empty TVP is actually bound.

        # pass a tuple as a single scalar parameter value. A TVP remains one
        # parameter by wrapping its rows in the ordinary outer parameter tuple:
        # execute("EXEC dbo.proc ?", (rows,))

mssql_python/pybind/ddbc_bindings.cpp:914

• The prepared IPD's catalog is read and cached above, but binding only writes SQL_CA_SS_SCHEMA_NAME; SQL_CA_SS_CATALOG_NAME is never propagated to the implementation descriptor. A call to a procedure in a non-current database can therefore resolve the TVP type against the wrong catalog even though metadata discovery succeeded. Set the catalog descriptor field when it is present (or pass a fully qualified type name).

        rc = SQLSetDescField_ptr(implementationDescriptor, recordNumber, SQL_CA_SS_SCHEMA_NAME,
                                 reinterpretU16stringAsSqlWChar(*schema),
                                 static_cast<SQLINTEGER>(schema->size() * sizeof(SQLWCHAR)));

• Files reviewed: 7/7 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Critical TVP binding issues must be resolved before approval.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

mssql_python/pybind/ddbc_bindings.cpp:569

• This condition rejects a precision above 38, but the message says the TVP column requires precision greater than 38, which tells the caller the opposite of the actual constraint. Please use wording consistent with the scalar validation (for example, that the maximum supported precision is 38) and update the new assertion that matches this text.

                throw py::value_error("TVP Decimal column requires precision greater than 38");

• Files reviewed: 8/8 changed files
• Comments generated: 3
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Native ODBC TVP and cursor-lifecycle changes warrant final human review, and two assessments require changes.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

mssql_python/pybind/ddbc_bindings.cpp:570

• This branch rejects precision above 38, but the message says the TVP column “requires precision greater than 38,” which communicates the opposite condition and can mislead callers diagnosing valid Decimal inputs. Please state that precision cannot exceed 38.

• Files reviewed: 8/8 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

The changes span Python, native ODBC bindings, metadata handling, and multiple execution paths requiring final human review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

mssql_python/pybind/ddbc_bindings.cpp:570

• This branch rejects a common source precision greater than 38, but the message says the column requires precision greater than 38, which reads as the opposite condition. Report that the TVP decimal column exceeds SQL Server's 38-digit precision limit so callers know what to change.

• Files reviewed: 8/8 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

The native ODBC TVP binding and metadata lifecycle require final human validation across supported platforms and drivers.

Review details

• Files reviewed: 8/8 changed files
• Comments generated: 0 new
• Review effort level: Lite