Skip to content

feat(consumergate)!: rebuild the gate on the postpone primitive - #492

Draft
behinddwalls wants to merge 1 commit into
preetam/remove-nack-delayfrom
preetam/gate-on-postpone
Draft

feat(consumergate)!: rebuild the gate on the postpone primitive#492
behinddwalls wants to merge 1 commit into
preetam/remove-nack-delayfrom
preetam/gate-on-postpone

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The gate parked blocked deliveries in memory: the partition goroutine sat in a watch/extend loop, renewing the parked delivery's visibility until the gate opened. That mechanism only babysat the one delivery it parked — messages already fetched into the per-partition buffer behind it (up to BatchSize) had their visibility lapse, redelivered as duplicates that stuffed the buffer until the topic's routing loop stalled, and burned a retry attempt per lapse until they were spuriously dead-lettered without ever failing. It also held a goroutine and a live lease per blocked partition for the whole close.

What?

The pre-Process gate check now postpones instead of parking: a blocked delivery gets its parked record written (observability) and is postponed for a re-check delay (~1s) — back to the queue as a partition barrier, redelivering without retry cost, re-checking the gate on each redelivery. Every buffered delivery behind a closed gate is postponed in turn as the partition drains, so nothing waits in memory, nothing lapses, and Stop never has parked state to unwind. The admit path unconditionally removes the parked record, which is what releases e2e awaitUnparked.

Contract: Entry.Watch and the Wait helper are replaced by Park/Unpark (record ops); Config/DefaultConfig and the never-implemented Factory are deleted — the file store no longer polls, since the re-check cadence is the consumer's postpone delay. Semantics changes: release is quantized to the re-check delay (same order as the old 1s file poll), and a released message is a fresh delivery (Attempt restarts at 1) rather than "the same attempt" — no test asserted either. Consumer-gate RFC, extension READMEs, and the consumer README are updated; in-memory parking moves to the RFC's Rejected list with the buffered-lapse hazard as the reason.

Test Plan

make test (83 targets) — reworked gate tests: blocked deliveries are parked + postponed with the re-check delay and never processed; redelivery after open unparks and processes; partition-scoped gating; Stop is immediate while gated; fail-open on read errors. ✅ bazel test //test/e2e/submitqueue/... — the cancel-caught-pre-batch scenario passes unchanged: awaitParked observes the record while closed, awaitUnparked observes its removal after open, the stale check is dropped, the sentinel lands. ✅ make fmt, make mocks.

Stack

  1. docs(rfc): consumer hold — postpone a delivery as a partition barrier #486
  2. feat(platform): hold/postpone — a fourth delivery outcome for backing off #487
  3. feat(stovepipe): migrate process and buildsignal waits to the hold primitive #488
  4. feat(orchestrator): migrate the buildsignal poll loop to the hold primitive #489
  5. feat(messagequeue)!: remove PublishAfter and the visible_after column #490
  6. feat(messagequeue)!: drop the requeue delay parameter from Nack #491
  7. @ feat(consumergate)!: rebuild the gate on the postpone primitive #492

## Summary

### Why?

The gate parked blocked deliveries in memory: the partition goroutine sat in a watch/extend loop, renewing the parked delivery's visibility until the gate opened. That mechanism only babysat the one delivery it parked — messages already fetched into the per-partition buffer behind it (up to BatchSize) had their visibility lapse, redelivered as duplicates that stuffed the buffer until the topic's routing loop stalled, and burned a retry attempt per lapse until they were spuriously dead-lettered without ever failing. It also held a goroutine and a live lease per blocked partition for the whole close.

### What?

The pre-Process gate check now postpones instead of parking: a blocked delivery gets its parked record written (observability) and is postponed for a re-check delay (~1s) — back to the queue as a partition barrier, redelivering without retry cost, re-checking the gate on each redelivery. Every buffered delivery behind a closed gate is postponed in turn as the partition drains, so nothing waits in memory, nothing lapses, and Stop never has parked state to unwind. The admit path unconditionally removes the parked record, which is what releases e2e awaitUnparked.

Contract: Entry.Watch and the Wait helper are replaced by Park/Unpark (record ops); Config/DefaultConfig and the never-implemented Factory are deleted — the file store no longer polls, since the re-check cadence is the consumer's postpone delay. Semantics changes: release is quantized to the re-check delay (same order as the old 1s file poll), and a released message is a fresh delivery (Attempt restarts at 1) rather than "the same attempt" — no test asserted either. Consumer-gate RFC, extension READMEs, and the consumer README are updated; in-memory parking moves to the RFC's Rejected list with the buffered-lapse hazard as the reason.

## Test Plan

✅ `make test` (83 targets) — reworked gate tests: blocked deliveries are parked + postponed with the re-check delay and never processed; redelivery after open unparks and processes; partition-scoped gating; Stop is immediate while gated; fail-open on read errors. ✅ `bazel test //test/e2e/submitqueue/...` — the cancel-caught-pre-batch scenario passes unchanged: awaitParked observes the record while closed, awaitUnparked observes its removal after open, the stale check is dropped, the sentinel lands. ✅ `make fmt`, `make mocks`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant