feat(ipc): request-id envelopes on every frame; responses in completion order - #25196
Open
charlielye wants to merge 1 commit into
Open
feat(ipc): request-id envelopes on every frame; responses in completion order#25196charlielye wants to merge 1 commit into
charlielye wants to merge 1 commit into
Conversation
charlielye
changed the base branch from
next
to
cl/ipc-reactor-disconnect-cleanup
August 13, 2026 13:35
charlielye
force-pushed
the
cl/ipc-envelope-ids
branch
from
August 14, 2026 10:06
ae4c3b5 to
9f5af45
Compare
charlielye
force-pushed
the
cl/ipc-envelope-ids
branch
from
August 14, 2026 10:48
9f5af45 to
2efd4ae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #25150 (contains its commit until it merges; review the head commit only).
What
Every ipc-runtime frame now carries a client-assigned request id, and clients correlate responses by id instead of position:
The server echoes the id on the response and sends responses in completion order — the reactor's reorder stash, its per-connection sequence counters, and the
drain_disconnected_clients()GC hook are all deleted.respond()pushes onto a plain completion queue; the reactor drains and sends. A slow request no longer delays a fast one's response on the same connection (previously the stash re-serialized release even though the wsdb scheduler completed reads concurrently).Ids are per-connection, random-start (so a stale frame left in a recycled MPSC-SHM ring slot by a previous occupant can never pair with a live call — largely defusing the known SHM stale-ring follow-up), with 0 reserved for server-initiated frames.
Why
The TXE flake investigation (#25150) established that positional correlation makes any stale/mispaired frame a silent decode-roulette. Ids make the whole mispairing class structurally impossible and detectable: a frame that pairs with nothing is now a loud
failAll(“response for unknown request id …”) instead of aconsole.warn— closing the original hardening ask.How (ipc-runtime only — codegen and all generated packages untouched)
IpcServer::receive(client, uint64_t& id)/send(client, id, …);run()echoes internally.IpcClientgains explicit-idsend/receivevirtuals plus serial convenience overloads (send(data,…)/receive(t)) that auto-assign ids and verify the echo — keeping the generated C++ clients and every FFI binding source-compatible. Pipelined callers must use the explicit-id API (the reactor tests do).IpcClient::may_have_stale_frames()): SHM rings persist across occupants, so a frame addressed to a previous occupant's id is an anticipated leftover — the serial receive releases it and keeps waiting for the real response, and the TS async client discards it with a warning. Over UDS there is no reuse, so a foreign id is a genuine desync and fails loudly (serial: close; TS:failAll).ring_send_msg/ring_receive_msgcarry the id inside the ring message), napi async glue (call(id: bigint, buf), callback delivers(id, buffer)).UdsIpcClientandNapiShmAsyncClientholdMap<bigint, pending>(random-start ids);UdsIpcServerparses/echoes.IpcClientAsync.call()signature unchanged — consumers are untouched (CdbIpcServer wrapsUdsIpcServer, so it rides along).ipc_server_receive/sendgain the id parameter; the handler-loop APIs (ipc_server_run*) and the serial client calls are unchanged — so the rust and zig bindings need zero changes (verified: they bind only those surfaces).Version-mismatch story
This is a wire-protocol break; binaries and TS packages must move in lockstep (they already ship as matched pairs). Mismatches fail fast with explicit messages, both directions: a frame shorter than the id field is rejected as “IPC protocol mismatch (envelope ids); update the peer binary/package” (client-side
failAll, server-side disconnect + log). Labs-side note: the acvm/wsdb npm + toolchain artifacts pick this up on their next republish pinned to a commit containing this change; the rust transport there is the same FFI crate, so no code change — just the lockstep bump.Verification
ipc_runtime_tests21/21 release and 21/21 ASAN — includingSerialClientSkipsStaleFrameFromPreviousOccupant(SHM: injected stale frame is skipped, real response delivered) andSerialClientClosesOnForeignFrame(UDS: same injection fails the call). Pipelined tests converted from FIFO-order assertions to id-pairing assertions (each request answered exactly once); the reactor tests now also assert responses arrive out of send order under reversed-sleep handlers — pinning the head-of-line-blocking removal — and the MPSC test keeps its lost-wake stall probe.ipc-runtime/tssuite 18/18.aztec-wsdb+ new TS client running the world-state suites — sigpipe blast-radius test, churn legs A/B/C/D with 60s soaks (~1.2M id-paired calls, ~1.4k mid-flight-disconnect churn rounds), and the existing pipelined-read correlation suite over both transports — all green.