fix: keep long MCP tool calls alive before host timeout

[sperwe]

Problem

Long-running MCP tool calls can remain silent until the final tool result is ready. In the real Mac mini production path through Cloudflare Tunnel, several successful /mcp requests completed near the apparent client timeout boundary:

• 12,023 ms
• 11,663 ms
• 9,096 ms
• 12,027 ms

The host UI can remain at “Waiting for a tool result” and then report that sending the message timed out, even though DevSpace and the tunnel remain healthy.

Current diagnosis

The modern MCP handler uses its default response mode, so an ordinary tool call may not establish SSE before the result completes. exec_command waits 10 seconds by default and permits a 12-second yield window. This creates a long time-to-first-byte window that is close to the observed client timeout.

Current request logging is completion-only, which makes a stuck or client-aborted request invisible. Aggregate narrow-tool calls also log their internal child calls without a parent identifier, and a batch with running children can currently be logged as successful.

Proposed scope

1. Establish the OpenAI-facing modern MCP response as SSE and emit periodic keepalive traffic.
2. Reduce the default process yield window to 3 seconds while retaining the existing explicit maximum.
3. Add request lifecycle observability for request start, first byte or stream establishment, completion, and client abort.
4. Make aggregate/child tool-call relationships and running outcomes unambiguous in logs.
5. Add regression coverage for time-to-first-byte, long-running command session return, request abort, and aggregate logging semantics.

Acceptance

• A silent command longer than the default yield returns a running process session within the client-safe window.
• The public MCP transport establishes a streaming response promptly and keeps it active during long work.
• Client disconnects are visible in logs and do not appear as successful completion.
• Running aggregate operations are not reported as terminal success.
• Existing MCP compatibility, process-session, typecheck, tests, and build remain green.

Non-goals

• Changing Cloudflare Tunnel ownership or configuration.
• Auto-cancelling every detached process without an explicit lifecycle policy.
• Broad redesign of all long-running tools in this patch.

[sperwe]

Implementation specification

Classification: P0 reliability, high certainty, focused compatibility fix, estimated under three days. Existing transport/process primitives are already present; this change closes specific reliability and observability gaps.

Root cause

The modern OpenAI-facing MCP handler currently uses its default response mode, so a tool call without progress notifications can remain silent until its final JSON result. At the same time, process execution waits 10 seconds by default before returning a running session. Together these place time-to-first-byte near the observed host timeout boundary. Request logging is completion-only, and narrow aggregate logging treats failed === 0 as success even when children remain running; child calls also lack a stable aggregate correlation key.

Required changes

1. Configure the modern createMcpHandler path for SSE response mode with an approximately five-second keepalive, while preserving the existing legacy stateless compatibility path.
2. Reduce the default process execution yield from 10,000 ms to 3,000 ms. Preserve explicit yields through the current 12,000 ms maximum; omitted yields in run_commands_parallel and compatibility batches inherit the new process default.
3. Add safe HTTP/MCP lifecycle telemetry for request start, first response or stream establishment, completion, and client abort, correlated by request ID. Do not log credentials, authorization headers, bodies, commands, or tool arguments.
4. Add stable aggregate/child correlation (aggregate ID plus child index) to narrow aggregate tool logs. Running aggregates must not be reported as terminal success; failure and completion remain distinguishable.
5. Add focused regressions for forced SSE/keepalive at the handler-testable layer, silent commands longer than three seconds returning a running session promptly, abort logging, and aggregate running/correlation semantics.

Compatibility and non-goals

• Keep legacy MCP compatibility and existing response schemas intact.
• Keep the explicit 12-second process yield ceiling and existing write_stdin polling behavior.
• Do not change tunnel ownership/configuration or introduce blanket process cancellation on disconnect.
• Keep the patch small; grep sibling aggregate/process paths when changing shared behavior.

Verification

Run focused MCP transport, server, and process-session tests; then typecheck; then full test/build if focused checks remain green. No commit, push, PR, merge, or deployment is part of this issue pass.