Skip to content

InternalFailure: "Output pipe not initialized" when a write races connection teardown #3167

Description

@mgravell

Summary

A write that reaches PhysicalBridge.WriteMessageToServerInsideWriteLock while the underlying
connection is being torn down and re-established fails with an InvalidOperationException
("Output pipe not initialized"), surfaced to the caller as
RedisConnectionException: InternalFailure. Because it is an InternalFailure rather than a
recognised connection failure, it is not subject to the usual backlog/retry handling.

Seen intermittently; @mgravell reports having hit it before, so this is not a one-off.

Trace

StackExchange.Redis.RedisConnectionException : InternalFailure on [0]:UNWATCH (TransactionProcessor)
---- System.InvalidOperationException : Output pipe not initialized

----- Inner Stack Trace -----
   at StackExchange.Redis.PhysicalConnection.<get_Output>g__Throw|172_0() in /_/src/StackExchange.Redis/PhysicalConnection.Write.cs:line 18
   at StackExchange.Redis.PhysicalConnection.get_Output() in /_/src/StackExchange.Redis/PhysicalConnection.Write.cs:line 17
   at StackExchange.Redis.Message.WriteTo(PhysicalConnection physical) in /_/src/StackExchange.Redis/Message.cs:line 950
   at StackExchange.Redis.PhysicalBridge.WriteMessageToServerInsideWriteLock(PhysicalConnection connection, Message message) in /_/src/StackExchange.Redis/PhysicalBridge.cs:line 1612

What provoked it

Observed during a full test-suite run, in TransactionTests.BasicTranWithStreamLengthCondition.
The relevant detail is what was happening on the shared connection at that moment: a parallel
ClientKillTests.ClientKillWithMaxAge had just issued a CLIENT KILL ... MAXAGE, killing the
interactive connection, and the bridge was mid-backlog-drain when the reconnect happened.

From the same test's output, immediately before the failure:

Interactive/127.0.0.1:6379: MessagesSinceLastHeartbeat: 317, ConnectedAt: 2026-08-06 14:30:14Z,
  Writer: Active, BacklogStatus: WritingMessage,
  BacklogMessagesPending: (Queue: 193, Counter: 193), TotalBacklogMessagesQueued: 369,
  Connection: (SentAwaitingResponse: 0, AvailableOnSocket: 0 byte(s), InReadPipe: 0 byte(s),
               InWritePipe: -1 byte(s), ReadStatus: ReadAsync, WriteStatus: Writing)

and from the parallel test that did the killing:

Connect took: 3025ms
Connection Failed (Interactive,SocketClosed): ... SocketClosed on 127.0.0.1:6379/Interactive,
  Idle/RanToCompletion, last: CLIENT, origin: ReadAllAsync, ... v: 3.1.12.21783
Connection Restored (Interactive,None):

So: connection killed out from under an active backlog drain, and a queued message is then written
to a PhysicalConnection whose output pipe has already gone.

Expected

A write losing its connection mid-flight is an ordinary connection failure, and should be handled
as one (backlog per BacklogPolicy, or a RedisConnectionException describing the disconnect),
not an InternalFailure carrying an InvalidOperationException.

Environment

  • main @ 5517cbf2
  • net10.0 (.NET 10.0.10), library reporting v: 3.1.12.21783
  • Linux, server = redislabs/client-libs-test image, Redis 8.9.241
  • Reproduction is not reliable; killing connections while a backlog is draining looks like the way in.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions