docs(rfc): consumer hold — postpone a delivery as a partition barrier - #486
Draft
behinddwalls wants to merge 1 commit into
Draft
docs(rfc): consumer hold — postpone a delivery as a partition barrier#486behinddwalls wants to merge 1 commit into
behinddwalls wants to merge 1 commit into
Conversation
## Summary ### Why? Queue controllers have no way to say "this message is fine, but it must wait." The outcome model is ack/nack/reject, so the stovepipe process and buildsignal stages fake waiting by acking and republishing fresh copies of their own messages with delayed visibility. That workaround makes consumers publishers to their own topic, requires per-stage message-id minting to dodge the publish dedup (one scheme already shipped a stalled-loop bug), resets the delivery attempt count every cycle so the DLQ backstop never fires, hangs the loop's only liveness on a publish succeeding, and churns a row per wait tick. The process stage RFC weighed the alternatives, chose neither, and deferred to a future consumer-owned primitive — this is that primitive. ### What? Design doc for a hold/postpone primitive. A controller records a hold on its delivery view and returns success; the framework postpones the delivery instead of acking. The postponed message becomes a barrier: its partition is not consumed past it until the delay elapses, then it redelivers first, in order — a backoff loop that costs no goroutine, no lease renewal, and no new rows between wake-ups. A postponed redelivery resets failure accounting (a postponed mark on the delivery state exempts the next delivery from the attempt increment), so holds never burn the dead-letter budget, while real failures still do. Failed postpone writes lapse into normal visibility-timeout redelivery, so wait-loop liveness is framework-owned. Rejected alternatives (park-and-extend, non-blocking postpone, sentinel errors, gate write path, framework hold caps) and stovepipe/orchestrator migrations as named follow-ups.
behinddwalls
force-pushed
the
preetam/consumer-backoff
branch
from
August 1, 2026 17:22
d7dfedb to
1578ec0
Compare
This was referenced Aug 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
Queue controllers have no way to say "this message is fine, but it must wait." The outcome model is ack/nack/reject, so the stovepipe process and buildsignal stages fake waiting by acking and republishing fresh copies of their own messages with delayed visibility. That workaround makes consumers publishers to their own topic, requires per-stage message-id minting to dodge the publish dedup (one scheme already shipped a stalled-loop bug), resets the delivery attempt count every cycle so the DLQ backstop never fires, hangs the loop's only liveness on a publish succeeding, and churns a row per wait tick. The process stage RFC weighed the alternatives, chose neither, and deferred to a future consumer-owned primitive — this is that primitive.
What?
Design doc for a hold/postpone primitive. A controller records a hold on its delivery view and returns success; the framework postpones the delivery instead of acking. The postponed message becomes a barrier: its partition is not consumed past it until the delay elapses, then it redelivers first, in order — a backoff loop that costs no goroutine, no lease renewal, and no new rows between wake-ups. A postponed redelivery resets failure accounting (a postponed mark on the delivery state exempts the next delivery from the attempt increment), so holds never burn the dead-letter budget, while real failures still do. Failed postpone writes lapse into normal visibility-timeout redelivery, so wait-loop liveness is framework-owned. Rejected alternatives (park-and-extend, non-blocking postpone, sentinel errors, gate write path, framework hold caps) and stovepipe/orchestrator migrations as named follow-ups.
Test Plan
Issues
Stack