Enable ConnectionPoolV2 by default

[mdaigle]

CI results summary

This PR flips UseConnectionPoolV2 to true so the full CI matrix runs against ChannelDbConnectionPool by default. CI exposed these V1/V2 differences and latent test issues:

1. Emancipated connection reclamation: ReclaimEmancipatedOnOpenTest exposed missing reclaim-on-open behavior in V2. The fix merged through Channel Pool: Reclaim leaked connections #4529 and is now on main.
2. TVP packet-wraparound test: TvpTest.TestPacketNumberWraparound exposed a latent Task<Task> test bug. Commit 6941f0c63 unwraps the task and propagates early failures correctly.
3. Physical-open timeout errors: AmbientTransactionFailureTest exposed a V2 timeout race on Windows. Commit 00e17ff4f preserves the physical connection error instead of replacing it with a pool-exhaustion timeout.
4. Simulated failover isolation: commit d0e0ae2c2 isolates pool-group metadata for ephemeral ports and makes login-token assertions independent of fatal connection-break timing.
5. Stress-worker exception handling: build 168916 exposed async void workers that terminated the test host when SQL Server reset connections. Commit 680e297fd observes all worker tasks and reports their exceptions through the test. It also quarantines the synchronous variant of the existing simulated transient-retry timing flake after the same failure appeared on Windows x86.
6. Failover pool-clear assertion: project build 168925 reproduced a previously documented abandoned pre-login race. Commit 87ccff741 verifies completed failover logins, preserving the fresh-connection contract without treating abandoned transport attempts as extra physical connections.
7. Simulated network-delay budget: package build 168942 spent 4.6 seconds in TLS handshake before the test's intentional 1-second delay exhausted its 5-second connection timeout. Commit 6c3ee9b7e widens the sync and async retry-disabled test budgets to 10 seconds. These tests verify retry behavior, not timeout precision.
8. Simulated failover timing: package build 168968 reproduced the same CI-load timeout as the already quarantined retry-disabled failover test. Commit 07bb497ae quarantines the retry-enabled sibling.
9. Connection resiliency after pool removal: unified build 168981 exposed a V2 replacement gap. A broken connection can be removed from the pool before reconnect asks for its replacement, leaving no old slot to swap. Commit 45b28ed95 acquires through the normal pool path when the old connection is already detached and adds deterministic max-pool-size coverage.

The clean matrix on 07bb497ae produced:

• Project build 168979: one macOS SQL Server 2025 container crash before tests.
• Unified build 168981: ConnectionResiliencySPIDTest exposed the detached-old-connection V2 regression corrected by 45b28ed95.
• Remaining jobs were canceled by the follow-up push.

The clean matrix on 45b28ed95 completed all 366 checks:

• Unified build 168998 succeeded.
• Project build 168996 had two unrelated infrastructure failures: a macOS SQL Server container terminated its connection and remained unavailable through retry, and an ARM64 Azure SQL host forcibly closed TLS logins before later command timeouts. All other project checks passed.
• Package build 168997 had two unrelated infrastructure failures: a macOS SQL Server 2025 container crashed during startup before tests, and the ARM64 Azure job reported Identity not found for its managed identity. All other package checks passed.
• GitHub reports 360 passing checks and four failed leaf jobs. The two additional failed checks are their project and package aggregates. No reproducible product regression remains.

Summary

Changes the default connection pool from WaitHandleDbConnectionPool to ChannelDbConnectionPool. Applications can restore the legacy pool explicitly:

AppContext.SetSwitch("Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2", false);

Changes

• Changes UseConnectionPoolV2's default to true.
• Updates the switch documentation and default-value unit test.
• Fixes TvpTest.TestPacketNumberWraparound so Task.Factory.StartNew's nested task is unwrapped and observed.
• Preserves physical connection failures when the caller's timeout expires immediately before V2 starts physical connection creation.
• Adds deterministic expired-timeout physical-open coverage.
• Isolates simulated failover tests from stale ephemeral-port pool-group metadata and fatal-error timing.
• Observes connection-pool stress workers so failures reach xUnit instead of crashing the test host.
• Quarantines CI-load simulated retry and failover timing tests.
• Verifies completed logins in the failover pool-clear test so abandoned pre-login attempts do not make the physical-connection count flaky.
• Gives simulated network-delay tests enough budget for CI host TLS setup before their intentional delay.
• Reacquires normally when connection resiliency has already removed the broken connection's pool slot.

Performance comparison

An interleaved best-of-three comparison covered 162 benchmarks with a 10% threshold:

• 41 improvements, primarily under higher-concurrency pool stress.
• 9 confirmed regressions.
• 3 unconfirmed regressions.

Confirmed regressions

Benchmark	Method	Parameters	Baseline	V2 default	Delta
SqlConnectionRunner	OpenAsyncConnection	MARS=True, Pooling=True	0.0035 ms	0.0087 ms	+145.23%
ConnectionPoolContentionRunner	SteadyStateOpenQueryClose	Parallelism=50, MaxPoolSize=10	40.2628 ms	98.2759 ms	+144.09%
SqlConnectionRunner	OpenAsyncConnection	MARS=False, Pooling=True	0.0036 ms	0.0076 ms	+108.42%
ConnectionPoolStressRunner	RapidFireOpenClose	Parallelism=20, MaxPoolSize=50	13.5875 ms	21.5105 ms	+58.31%
ConnectionPoolStressRunner	RapidFireOpenClose	Parallelism=25, MaxPoolSize=100	15.4399 ms	23.5592 ms	+52.59%
ConnectionPoolStressRunner	RapidFireOpenClose	Parallelism=25, MaxPoolSize=50	14.9726 ms	22.5136 ms	+50.37%
ConnectionPoolStressRunner	RapidFireOpenClose	Parallelism=10, MaxPoolSize=50	11.4258 ms	16.3225 ms	+42.86%
ConnectionPoolStressRunner	RapidFireOpenClose	Parallelism=20, MaxPoolSize=100	13.4755 ms	18.1328 ms	+34.56%
ConnectionPoolStressRunner	RapidFireOpenClose	Parallelism=10, MaxPoolSize=100	13.1124 ms	16.1011 ms	+22.79%

The strongest remaining performance signals are fixed per-call overhead in pooled OpenAsync, allocation and synchronization cost in rapid open/close loops, and contention when MaxPoolSize is small relative to concurrency. Higher-concurrency workloads with adequately sized pools generally improved.

Validation

• ChannelDbConnectionPoolTest: 64/64 passed on net8.0, net9.0, and net10.0 before the rebase; 64/64 passed on net9.0 after rebasing onto main.
• ChannelDbConnectionPoolReplaceConnectionTest: 16/16 passed on net8.0 and net9.0 after adding detached-old-connection coverage.
• AmbientTransactionFailureTest: 2/2 passed locally on net8.0 and net9.0 before the rebase; 2/2 passed on net8.0 after rebasing.
• Default-switch and connection-pool unit selection: 332/332 passed on net8.0, net9.0, and net10.0.
• The seven CI legs affected by TvpTest.TestPacketNumberWraparound passed after the test fix.
• The sync transient-retry theory passes locally on net9.0: 3/3.
• NetworkError_TriggersFailover_ClearsPool passes locally on net9.0.
• NetworkDelay_RetryDisabled sync and async theories passed 40/40 executions across net8.0 and net9.0.
• The quarantined retry-enabled failover test passes locally on net8.0.
• Manual test project builds on net8.0 with no warnings.

Suggested release note

Changed the default connection pool implementation to the Channel-based pool. Set Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2 to false to restore the legacy pool.

Checklist

• Tests added or updated
• Internal switch documentation updated
• No public API changes
• Full CI matrix completed with only unrelated infrastructure failures

[Copilot]

Pull request overview

This PR flips the default of the internal Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2 AppContext switch to true, making the Channel-based pool (ChannelDbConnectionPool) the default connection pooling implementation while keeping the legacy pool (WaitHandleDbConnectionPool) available via explicit opt-out.

Changes:

• Change LocalAppContextSwitches.UseConnectionPoolV2 default from false → true, and update its XML doc accordingly.
• Update the unit test that asserts the default switch values.
• Update the internal feature documentation table for the switch default.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/LocalAppContextSwitches.cs	Flips UseConnectionPoolV2 default to true and updates XML documentation.
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/LocalAppContextSwitchesTest.cs	Updates default-value assertion to expect UseConnectionPoolV2 == true.
.github/instructions/features.instructions.md	Updates the documented default for UseConnectionPoolV2 in the AppContext switches table.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mdaigle]

CI investigation: the two failing legs (sqlclient_manual_azure_123_linux_net10, sqlclient_manual_azure_123_windows_net10) plus a third failing leg (sqlclient_manual_azure_123_linux_net9, currently showing as part of the check group) all fail on the same test: ConnectionPoolTest.ReclaimEmancipatedOnOpenTest, with InvalidOperationException: Timeout expired ... obtaining a connection from the pool.

This is not a pre-existing flake: the same test passes on #4504's CI (same base branch, UseConnectionPoolV2 still defaulting to false there). It only fails here because this PR flips the default to true, routing the test through ChannelDbConnectionPool.

Root cause: WaitHandleDbConnectionPool (v1) has an explicit ReclaimEmancipatedObjects() path invoked when the pool is exhausted, letting Open() reclaim a GC'd-but-undisposed connection instead of timing out. ChannelDbConnectionPool (v2) has no equivalent active reclamation path (only a comment referencing IsEmancipated), so it can't reclaim the emancipated connection and Open() times out waiting for a pool slot.

Per this PR's stated purpose, I'm not fixing pool internals here — documented the gap in the PR body so the pool-v2 workstream can add reclaim-on-open support to ChannelDbConnectionPool before this default flip ships.

[mdaigle]

Broader CI update: failures across nearly the full manual-test matrix

As more legs of sqlclient-pr completed, the failure count grew beyond the initial 2-3 legs. Pulled the full test-execution logs for every failing sqlclient_manual_*_123_* leg (azure and localhost variants, net8/net9/net10/net462, Linux and Windows) from build 166919. Two distinct test failures account for all of them — no other tests are failing:

1. ConnectionPoolTest.ReclaimEmancipatedOnOpenTest — fails on every leg (azure + localhost, all TFMs/OSes)

Same root cause already documented in the PR description: ChannelDbConnectionPool (V2) has no active reclaim-on-open path for "emancipated" (GC'd-without-Dispose) connections, unlike WaitHandleDbConnectionPool (V1)'s ReclaimEmancipatedObjects(). Open() times out instead of reclaiming the slot. Confirmed via #4504's own CI (same base branch, switch still false) that this test passes there — this is caused by the default flip, not flakiness.

2. TvpTest.TestPacketNumberWraparound — fails on every localhost leg only (not azure legs, since it's IsNotAzureServer-gated)

This is a distinct, pre-existing regression test for a specific TdsParserStateObject.WritePacket byte-counter-wraparound bug. It opens its own SqlConnection against the default (no MaxPoolSize override) TCP connection string via OpenAsync, then drives a custom 1,000,000-row TVP enumerator through ExecuteNonQueryAsync (swallowing errors from the sproc/table-type not existing, by design — the test only cares whether the full row-set gets enumerated before that error hits).

Observed failure signature across all 5 localhost logs is consistent and non-random: the enumerator only advances 1-18 elements out of 1,000,000 in 1.5-6.2 milliseconds (not the 60s timeout), e.g.: enumerator.Count=3, enumerator.MaxCount=1000000, elapsed=00:00:00.0062206

That signature (near-zero elapsed time, tiny count) points to the task faulting essentially immediately, most likely during connection.OpenAsync(cancellationToken), which is not wrapped in the test's try/catch (only ExecuteNonQueryAsync is), so any exception there (e.g. a pool-related timeout/error) would abort the task before any rows are read, matching what's observed.

I have not confirmed the exact exception (stdout only surfaces the assertion failure, not the swallowed/faulted exception detail), and I'm not certain this is the same root cause as #1. It may be a related knock-on effect (e.g. state left behind by the ReclaimEmancipatedOnOpenTest failure destabilizing the default connection pool for subsequent tests in the same run) or a separate, independent V2 pool gap. Flagging this clearly as a second, distinct finding rather than assuming it's explained by #1.

No other manual test failures were found in any of the ~15 failing legs beyond these two. Per this PR's scope, not attempting to fix pool internals — surfacing both findings here for the pool-v2 workstream to investigate before this default flip ships.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/instructions/features.instructions.md:257

• The AppContext switch default-value table is now inconsistent with the actual defaults in LocalAppContextSwitches.cs: UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni both default to true (compatibility mode), but the table still lists false. Since this PR already edits this section, please update these rows so the table reflects real defaults and explains that setting them to false enables the newer behaviors.

| `Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour` | `false` | Uses legacy async behavior for compatibility |
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni` | `false` | Uses legacy SNI processing path |
| `Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2` | `true` | Enables the new `ChannelDbConnectionPool` implementation; set to `false` to restore the legacy `WaitHandleDbConnectionPool` |

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/instructions/features.instructions.md:257

• The switch-default table still lists UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni as defaulting to false, but both are asserted as true defaults in LocalAppContextSwitchesTest and documented/implemented as defaultValue: true in LocalAppContextSwitches.cs. This table should be updated to avoid misleading contributors about the actual defaults.

| `Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour` | `false` | Uses legacy async behavior for compatibility |
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni` | `false` | Uses legacy SNI processing path |
| `Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2` | `true` | Enables the new `ChannelDbConnectionPool` implementation; set to `false` to restore the legacy `WaitHandleDbConnectionPool` |

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/instructions/features.instructions.md:257

• The switch default-value table is inconsistent with the actual defaults in LocalAppContextSwitches.cs: both UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni default to true (see LocalAppContextSwitches.cs:539-575), but this table still lists them as false. This makes the switch reference misleading, especially now that UseConnectionPoolV2 is being updated here as well.

| `Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour` | `false` | Uses legacy async behavior for compatibility |
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni` | `false` | Uses legacy SNI processing path |
| `Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2` | `true` | Enables the new `ChannelDbConnectionPool` implementation; set to `false` to restore the legacy `WaitHandleDbConnectionPool` |

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/LocalAppContextSwitches.cs:588

• The PR description calls out a missing “reclaim emancipated connections on open” path in PoolV2 as the primary root cause of CI failures, but ChannelDbConnectionPool.GetInternalConnection already performs a reclaim sweep on the slow path before it parks on the idle channel (see ChannelDbConnectionPool.cs:1516-1526, ReclaimEmancipatedConnections()). Either the investigation summary is out of date, or the problem is that reclamation is not triggering / not freeing a usable connection; please update the PR description (or add a note) so the documented root cause matches the current code.

    /// </summary>
    public static bool UseConnectionPoolV2 =>
        AcquireAndReturn(
            UseConnectionPoolV2String,
            defaultValue: true,

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.github/instructions/features.instructions.md:257

• The switch-default table now claims UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni default to false, but LocalAppContextSwitches defaults both to true (see LocalAppContextSwitches.cs:557 and :575). Since this PR updates this table, it should keep these defaults accurate to avoid misleading guidance.

| `Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour` | `false` | Uses legacy async behavior for compatibility |
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni` | `false` | Uses legacy SNI processing path |
| `Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2` | `true` | Enables the new `ChannelDbConnectionPool` implementation; set to `false` to restore the legacy `WaitHandleDbConnectionPool` |

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/TvpTest.cs:72

• Task.Factory.StartNew(..., LongRunning).Unwrap() is unnecessary here and adds thread-creation overhead; RunPacketNumberWraparound is already async and can be started directly and raced against the timeout task.

            // Task.Factory.StartNew with an async delegate returns a Task<Task>, so it must be
            // unwrapped before use in Task.WhenAny below. Without Unwrap(), WhenAny would observe
            // only the outer task (which completes as soon as the async lambda hits its first
            // await) instead of the actual completion of RunPacketNumberWraparound.
            Task actionTask = Task.Factory.StartNew(

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/TvpTest.cs:86

• If the timeout wins, any fault from actionTask is never observed (and the task may keep running briefly after test failure), which can hide the real failure cause and leak background work into later tests. Consider best-effort observing faults after cancellation on the timeout path.

            // Propagate any unexpected failure from the action task (e.g. a connection open
            // failure) instead of letting it surface only as a low enumerator count below.
            if (completedTask == actionTask)
            {
                await actionTask;

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTest.cs:2241

• The XML doc comment for CountingTimeoutConnectionFactory.CreateConnection says it always throws the pooled-open timeout, but the implementation now throws a supplied marker exception when provided. Update the comment to match the new behavior.

                TimeoutTimer timeout)
            {
                CreateCount++;
                throw _exception ?? ADP.PooledOpenTimeout();

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/TvpTest.cs:74

• Task.Factory.StartNew without specifying TaskScheduler.Default uses TaskScheduler.Current, which can be a non-default scheduler under test frameworks and lead to unexpected scheduling for a LongRunning task. Other tests in this repo pass TaskScheduler.Default explicitly (e.g., ChannelDbConnectionPoolWarmupTest.cs:190-204).

            Task actionTask = Task.Factory.StartNew(
                () => RunPacketNumberWraparound(enumerator, cancellationTokenSource.Token),
                TaskCreationOptions.DenyChildAttach | TaskCreationOptions.LongRunning).Unwrap();

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolStressTest.cs:199

• workers is allocated with ConcurrentConnections elements but when ConcurrentConnections == 1 no worker is ever assigned (the for loop does not run and the if (ConcurrentConnections > 1) block is skipped). This leaves a null element in the array and Task.WhenAll(workers) will throw (ArgumentException) before the stress test actually runs.

            if (ConcurrentConnections > 1)
            {
                workers[ConcurrentConnections - 1] = CreateWorkerTask(
                    connectionString, command, barrier, doomConnections: true, async, doomAction);
            }

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolStressTest.cs:184

• RunStressTest allocates a Task[] sized to ConcurrentConnections but does not populate any entry when ConcurrentConnections == 1 (the for-loop runs 0 iterations and the dooming worker is only created when > 1). This leaves a null in the array and causes Task.WhenAll(workers) to throw immediately, potentially leaving started workers unobserved.

            var workers = new Task[ConcurrentConnections];
            using Barrier barrier = new(ConcurrentConnections);

            var command = string.IsNullOrWhiteSpace(WaitForDelay)
                ? "SELECT GETDATE()"

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolStressTest.cs:184

• workers is sized to ConcurrentConnections, but when ConcurrentConnections == 1 neither the for loop nor the dooming-worker branch assigns workers[0], so Task.WhenAll(workers) will throw due to a null task. Either validate ConcurrentConnections >= 2 up front or populate all slots and conditionally mark one worker as the dooming worker.

            var workers = new Task[ConcurrentConnections];
            using Barrier barrier = new(ConcurrentConnections);

            var command = string.IsNullOrWhiteSpace(WaitForDelay)
                ? "SELECT GETDATE()"

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolStressTest.cs:191

• RunStressTest leaves workers[0] as null when ConcurrentConnections == 1 (the loop runs 0 iterations and the dooming worker is only created when > 1). Task.WhenAll(workers) will then throw ArgumentException/NullReferenceException instead of running the stress test. Create all worker tasks in a single loop and only enable the dooming behavior on the last worker when ConcurrentConnections > 1.

            // Create regular threads (don't doom connections)
            for (int i = 0; i < ConcurrentConnections - 1; i++)
            {
                workers[i] = CreateWorkerTask(
                    connectionString, command, barrier, doomConnections: false, async);