Shovels and federation

The two ways this cluster exchanges messages with another one — and they are not variants of each other.

A shovel is a one-way link the broker runs for you: you name a source and a destination, and it moves messages between them. Federation is policy-driven: you register an upstream, and a policy naming it creates one link per matched object.

Put the other way: a shovel is something you define, and federation is something a rule creates.

The screen has a tab for each, both scoped by the Vhost selector in the header.

Shovels

Each shovel shows what it is moving — source to destination — its acknowledgement mode, its prefetch, and its state. no-ack is marked, because it is the fastest mode and the only lossy one: anything in flight when the link drops is gone. A shovel that sets no prefetch shows a dash, meaning the plugin's own default applies. Forwarded, the number of messages the shovel has moved, starts hidden; the tab's Columns control turns it on, and it shows a dash where the plugin reports no count.

Three states are kept apart because they mean different things:

• Not reported — the shovel is defined, but the plugin is not reporting a state for it. That is not the same as stopped.
• Terminated — with the broker's own reason where it gave one.
• Running, blocked — running and currently held.

A shovel that has terminated can still say what it was moving, because BROKA reads the definition and the state separately and joins them. The plugin's state list carries no topology for a terminated shovel; the definition does.

Deleting the definition is what stops a shovel — there is no separate stop, and the confirmation says so rather than leaving you looking for one: messages already forwarded stay where they were delivered, and anything still in the source stops moving. The confirmation asks for a reason.

Restart on a shovel's row stops it and starts it again from its definition — what to do once the far end of a shovel that terminated has been fixed. The definition is not changed. A shovel configured in the broker's own file has Restart locked, with the reason: it is restarted where it is defined.

New shovel creates one in the selected virtual host, from a name, a source queue and a destination queue, the acknowledgement mode, and optionally a prefetch count, a delete-after rule (never, queue-length or a message count) and a reconnect delay. Leave a URI empty and it points at this broker. A name already used in that virtual host is refused rather than replaced: the dialog says so, and Edit on that shovel's row is the way to change it. Edit opens the same form with the name locked, and names any keys the shovel carries that the form does not model — those are kept as they are. Three kinds of shovel cannot be edited here, and their Edit is locked with the reason: one configured in the broker's own file rather than at runtime (which cannot be deleted from here either), one using AMQP 1.0, and one moving through an exchange rather than a queue.

A shovel started by a queue's Move messages appears here too, but only while it runs: it removes itself once it has moved the messages the queue held when it started. Deleting it before then stops the move. See Message operations.

Federation

Upstreams and links are two tables, deliberately apart: what this cluster could pull from, and what is federating right now.

An upstream on its own moves nothing. It is a stored connection, and a policy naming it is what creates the links. An upstream no policy names is marked unused — and the tooltip says that is configuration, not a fault.

A link that stays in starting is one that cannot reach its upstream and is retrying. That is worth knowing because a 4.3 broker carries no error text on a federation link, so "starting" is the symptom you get instead of a message. Where a broker does report a failure for a link, a warning icon on its row carries the text.

New upstream registers one from a name and its remote URI, with the acknowledgement mode, the maximum hops, and optional prefetch, reconnect delay, expiry, message TTL and remote exchange or queue. A name already used in that virtual host is refused, as a shovel's is. Creating one moves nothing until a policy names it — see Policies. Deleting an upstream leaves any policy that names it matching, and the links it fed stop; the confirmation says so, and reminds you to delete the policy too if the federation is meant to go away. It asks for a reason too.

New shovel, New upstream, Restart, Edit and delete appear only for someone allowed to change this cluster, and not in a read-only environment or on a read-only connection.

Masked messages are not copied

Where a data masking rule covers messages, a shovel or an upstream that would copy them is refused unless you may read them unmasked, and the refusal names the rule: the copy would be readable wherever it landed, and the rule would not follow it there. The check compares plain names only and errs towards refusing. A shovel from a queue is refused when a rule covers that queue, or while any exchange rule masks anything; one from an exchange, when a rule covers that exchange, or while any queue rule masks anything; and one whose source is an AMQP 1.0 address — like an upstream that names neither a queue nor an exchange — while any rule masks anything. What is checked is the definition as it would be stored, so editing only a shovel's prefetch is checked too. Someone who may read those messages unmasked defines shovels and upstreams as before.

Credentials never come back

An upstream's remote address is shown with its username and without its password.

The password is not masked — it is removed, in the broker service, before the value reaches the orchestrator, its logs, the wire or a browser's network panel. Masking would leave something a later write could send back as if it were real.

Editing does not need it back. When you edit a shovel or an upstream, the form keeps the URI stored on the broker by default — shown without its password — and the broker service re-sends the one it already holds, so the password never crosses the wire in either direction. Choose to replace it instead, and the new URI is required in full, password included. The create form says the same thing up front: a password you type is stored by the broker and never shown here again.

An address BROKA cannot parse is reported as absent rather than passed through, on the same reasoning: an unparsed URI is one this code cannot promise it cleaned.

What is recorded

Creating, editing, restarting and deleting a shovel, and creating, editing and deleting an upstream, are audited — and the entries are credential-free by construction: they record the source and destination objects, the acknowledgement mode and the hop limit, and never read the URI fields at all.

Alerting on shovels and links

An alert rule can watch a shovel — whether it is running, whether it is blocked, and the messages it has pending — or a federation link — whether it is running — picked by name from a list. Two connection-wide figures count the shovels and the links that are not running.

The three shovel states above map onto the figures plainly. Not reported and Terminated both read running = 0 — for an alert a shovel that has not started and one that has stopped are both not moving anything, though this screen keeps them apart. Running, blocked reads running = 1 and blocked = 1. A federation link in starting, in error or shut down reads running = 0.

A shovel or link that is removed resolves its open incident, saying why. The starter rules offer shovel not running and federation link not running where the broker has them, and an incident links back to this screen. Every metric is listed in Alert metrics.