Redis Streams in production: the pending entries list

A consumer group on a Redis stream has entries nobody is finishing. They are in the group's pending list: delivered to a consumer, never acknowledged, still in the stream. The question is whether they are in flight or stuck, and the list answers it with three columns — owner, idle time and delivery count. This is how to read them, and what each action on the screen does to the stream.

1 — Delivered is not acknowledged

Redis hands a stream entry to one consumer in a group and keeps a note of it until that consumer acknowledges it. The note is the pending entry. Until the acknowledgement arrives the entry belongs to that consumer, other consumers in the group will not be handed it, and the group's lag has already moved past it.

So a pending list is not a backlog in the queue sense. It is work that was handed out. A list that empties as fast as it fills is a healthy group. A list that holds the same ids for fifteen minutes is a consumer that took work and did not finish it — and the entry is still in the stream, untouched.

2 — Read the delivery count first

For a chosen stream and group the screen shows how many entries are pending, the oldest and newest of them, and how many consumers are holding them, with a count beside each consumer. Then the entries themselves: id, owner, idle time, deliveries.

An entry delivered once and still pending is normal work. An entry delivered five times and still pending is a message something keeps failing to process, and BROKA marks it — with the sentence that matters: claiming it again is a retry, not a fix. A message that fails deterministically will fail a sixth time for the next consumer that takes it. The signal is telling you to look at why, not to move it.

The list opens in the stream's own order rather than ranked by delivery count: a flagged entry is coloured where it sits, because its position among its neighbours is part of what you are reading. Every column sorts when you want the ranking instead.

3 — Two kinds of quiet

The stream's Consumer groups tab lists each group's consumers, and each carries two measures that are easy to confuse. As Redis reports them, Idle is the time since the consumer last asked for anything — a read or a claim, even one that returned nothing. Inactive is the time since it last actually received an entry.

A consumer idle for seconds and inactive for hours keeps asking and is given nothing — a healthy poller on a quiet stream rather than a stuck one, and the two cannot be told apart from idle alone. A consumer holding pending entries and idle for hours has stopped asking at all. On servers older than Redis 7.2 the second figure is not reported, and the console shows it as absent with the version as the reason, not as zero.

4 — Claiming is a decision, behind an interlock

Claiming moves ownership of entries to a consumer, so work abandoned by a process that died can be picked up by one that is alive. It is something a person does here, at a moment they chose: Claim for the entries they selected, or Sweep to a consumer for every entry idle past the threshold, in one bounded pass that says when more is left to run again.

The minimum idle time on both dialogs is not a filter. It is an interlock, and the dialog starts it at sixty seconds rather than leaving it off. Redis will not hand over any entry touched more recently than the threshold, which is what stops two operators recovering the same backlog from taking each other's work mid-flight — and setting it to zero turns that protection off.

The result says what happened. If fewer entries came back than you named, the console says how many and why — the rest were touched more recently than the idle threshold — rather than reporting a partial success as success. The list's own minimum-idle box narrows what you look at; it is a different control, and the hint under each says which is which.

On Redis 8.8 and later there is one more move beside claiming and acknowledging: Return to group hands the selected entries back to their group, owned by no consumer, still pending and still in the stream, for whichever consumer takes them next. It asks one question — what happens to each entry's delivery count — because that is the number this list reads as the signal: it stays as it is, goes back down by one when the failure was not the entry's fault, or is set to its maximum when the entry is bad, so a consumer that stops retrying after a number of deliveries will not take it again. On an older server it is shown greyed out, with the reason.

Claim before you clean up. Deleting a consumer from its group — the button on its row in the Consumer groups tab — leaves the entries it held in the stream, but they can no longer be claimed: the record that the consumer owed them goes with it.

5 — Acknowledging does not delete

"Acknowledge" reads like "discard", so the console says what it actually does, in the confirmation and in a standing note under the list. Acknowledging removes an entry from this group's pending list and from nowhere else. The entry stays in the stream. Other groups still see it. A range read still returns it. What you have said is this group is finished with it, not this never happened.

That is the right action for an entry a consumer processed and then died before acknowledging. It is the wrong action for the entry delivered five times: it makes the failure disappear from the list without anyone having looked at it.

When you mean both — this group is finished with the entry, and it should leave the stream — Ack & delete does it in one step on Redis 8.2 and later. It carries the same consumer-groups choice as a trim, which says what the other groups reading the stream keep: their pending references, nothing, or the entry itself until every group has acknowledged it. On an older server it is shown greyed out, with the reason.

6 — When lag says unknown

On the Consumer groups screen a group's row carries the stream it reads, its consumers, its pending count, the id it last delivered and its lag. Groups are read per stream, because Redis has no command that lists them all, and the footer says how many streams were read.

Lag can be unknown, and that is a real state rather than a missing value. When entries the group had not yet read have been trimmed away, the arithmetic has no answer and Redis returns nothing. BROKA shows that as unknown. It does not show 0, because zero means caught up — the one claim that would be false in exactly the situation where the truth matters.

Moving the group does not bring those entries back. Set position, on the group's row in the stream's Consumer groups tab, moves where the group reads from — to $, the end of the stream, to 0, the beginning, or to an id you give. Moving it back delivers again every entry after the new id; moving it forward skips the entries in between for good. Either way the group's pending list is left as it is: what was delivered and not acknowledged is still owed. New group asks the same question once, as its starting point, with $ — only what arrives from now on — as the default.

7 — Watching without consuming

The Live tab follows a stream as entries arrive, and it reads without consuming: BROKA issues Redis's plain read, not the consumer-group read. Following a stream creates no pending entry, joins no group and changes no application's backlog — you can watch a production stream during an incident without becoming part of it. The tail stops by itself, says why, and offers to resume from the last id it saw.

What trimming does to a pending list

What happens to a pending list depends on the server. Before Redis 8.2, trimming a stream and deleting entries consult no consumer group: an entry a group had been delivered and not yet acknowledged is removed anyway, the pending list keeps the id, the entry behind it is gone, and a later claim on it returns nothing. From Redis 8.2 both dialogs ask what the groups should keep — their pending references, nothing, or only entries every group has already acknowledged — and a delete's result says how many entries stayed because a group still needed them. On an older server the choice is shown, greyed out, with the reason. Destroying a group discards its pending list; the entries survive, because they belong to the stream.

What is recorded, and what the environment allows

Acknowledging, acknowledging and deleting, claiming, sweeping, returning entries to the group, trimming, deleting entries, moving a group's position, deleting a consumer and destroying a group are audited, and the entries record what the write did to work in flight, not just that it happened — including, for a trim or a delete, the consumer-group choice it was made with, and for a deleted consumer, how many pending entries it took with it. Opening the live tail is recorded too under the default read auditing, as a read, with the note that it reads without consuming.

Every one of those writes is refused in a read-only environment, for administrators as well, and the refusal is itself recorded. A connection frozen with its own read-only switch refuses them even inside an open environment. The destructive ones — trimming, deleting entries, acknowledging and deleting, moving a group's position, deleting a consumer and destroying a group — need a Reason before they run, in every environment; a guarded environment asks for one on every write, including claims, sweeps, acknowledgements and returns to the group. The Redis page covers the rest of the Streams surface.

What this article does not cover

• Automatic recovery. Claiming and sweeping here are done by a person, when they choose to. If your consumers reclaim abandoned work on their own, that is your application's logic; this list shows its result.
• Throughput. The counts are measured when you ask, and nothing is kept between readings. A rate is a comparison you make with two readings and a clock.
• The consumer's own code. A delivery count of five names the entry. The reason it fails is in the consumer.

Try it yourself

The redis-pending lab starts Redis with a stream whose pending entries list is already stuck — entries delivered five and six times, idle consumers, a group whose lag reads unknown — so you can claim, sweep and set positions yourself.

Applies to BROKA 1.0 · Redis · Commercial.