Skip to content

docs(rfc): consumer hold — postpone a delivery as a partition barrier - #486

Draft
behinddwalls wants to merge 1 commit into
mainfrom
preetam/consumer-backoff
Draft

docs(rfc): consumer hold — postpone a delivery as a partition barrier#486
behinddwalls wants to merge 1 commit into
mainfrom
preetam/consumer-backoff

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Queue controllers have no way to say "this message is fine, but it must wait." The outcome model is ack/nack/reject, so the stovepipe process and buildsignal stages fake waiting by acking and republishing fresh copies of their own messages with delayed visibility. That workaround makes consumers publishers to their own topic, requires per-stage message-id minting to dodge the publish dedup (one scheme already shipped a stalled-loop bug), resets the delivery attempt count every cycle so the DLQ backstop never fires, hangs the loop's only liveness on a publish succeeding, and churns a row per wait tick. The process stage RFC weighed the alternatives, chose neither, and deferred to a future consumer-owned primitive — this is that primitive.

What?

Design doc for a hold/postpone primitive. A controller records a hold on its delivery view and returns success; the framework postpones the delivery instead of acking. The postponed message becomes a barrier: its partition is not consumed past it until the delay elapses, then it redelivers first, in order — a backoff loop that costs no goroutine, no lease renewal, and no new rows between wake-ups. A postponed redelivery resets failure accounting (a postponed mark on the delivery state exempts the next delivery from the attempt increment), so holds never burn the dead-letter budget, while real failures still do. Failed postpone writes lapse into normal visibility-timeout redelivery, so wait-loop liveness is framework-owned. Rejected alternatives (park-and-extend, non-blocking postpone, sentinel errors, gate write path, framework hold caps) and stovepipe/orchestrator migrations as named follow-ups.

Test Plan

Issues

Stack

  1. @ docs(rfc): consumer hold — postpone a delivery as a partition barrier #486
  2. feat(platform): hold/postpone — a fourth delivery outcome for backing off #487
  3. feat(stovepipe): migrate process and buildsignal waits to the hold primitive #488
  4. feat(orchestrator): migrate the buildsignal poll loop to the hold primitive #489
  5. feat(messagequeue)!: remove PublishAfter and the visible_after column #490
  6. feat(messagequeue)!: drop the requeue delay parameter from Nack #491
  7. feat(consumergate)!: rebuild the gate on the postpone primitive #492

## Summary

### Why?

Queue controllers have no way to say "this message is fine, but it must wait." The outcome model is ack/nack/reject, so the stovepipe process and buildsignal stages fake waiting by acking and republishing fresh copies of their own messages with delayed visibility. That workaround makes consumers publishers to their own topic, requires per-stage message-id minting to dodge the publish dedup (one scheme already shipped a stalled-loop bug), resets the delivery attempt count every cycle so the DLQ backstop never fires, hangs the loop's only liveness on a publish succeeding, and churns a row per wait tick. The process stage RFC weighed the alternatives, chose neither, and deferred to a future consumer-owned primitive — this is that primitive.

### What?

Design doc for a hold/postpone primitive. A controller records a hold on its delivery view and returns success; the framework postpones the delivery instead of acking. The postponed message becomes a barrier: its partition is not consumed past it until the delay elapses, then it redelivers first, in order — a backoff loop that costs no goroutine, no lease renewal, and no new rows between wake-ups. A postponed redelivery resets failure accounting (a postponed mark on the delivery state exempts the next delivery from the attempt increment), so holds never burn the dead-letter budget, while real failures still do. Failed postpone writes lapse into normal visibility-timeout redelivery, so wait-loop liveness is framework-owned. Rejected alternatives (park-and-extend, non-blocking postpone, sentinel errors, gate write path, framework hold caps) and stovepipe/orchestrator migrations as named follow-ups.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant