JMX metrics

Every Kafka broker and controller measures its own health, load and JVM and publishes the figures over JMX. With JMX set up on a connection, BROKA reads a fixed list of them from every node — in both editions, at no extra cost — and shows them where you already look: the Brokers page, its Graphs and Quorum tabs, each broker's detail, a topic's page and the alert rules. Nothing is stored. Setting JMX up is described on Brokers.

What is read

BROKA reads these figures and no others. The list is fixed in the product rather than configurable: a JMX agent answers whatever it is asked, and a console that could be pointed at any bean could be pointed at one that does something.

Group	Figures	Per
Health	whether the node is the active controller; offline partitions and leader and unclean elections, from the active controller; how far behind the cluster metadata the node is; partitions under and at their minimum in-sync replicas; ISR shrinks and expands per second; offline log directories; the broker's state	node, broker
Requests	produce, consumer-fetch and follower-fetch requests per second, and the mean and 99th-percentile time of each; failed produce and fetch requests per second; bytes rejected per second	broker
Utilisation	how idle the request handlers and the network threads are, as a percentage; the request and response queues; the produce and fetch purgatories	broker
Disk	log flushes per second and their 99th-percentile time	broker
Throughput	bytes in and out and messages in per second	broker
JVM and system	heap used and its maximum; garbage collections and the time they take, per second; CPU; open file descriptors and the limit; threads	node
One topic	bytes in and out and messages in per second, from the brokers that lead its partitions	topic, on demand

Rates are the one-minute rate Kafka keeps; a time is the mean or the 99th percentile; the idle figures, which Kafka reports as fractions, are shown as percentages; garbage collection is a rate between two readings, so it appears from the second one.

The request handlers' idle figure is the broker's own pool's on Kafka 4. On an older Kafka, a node that is both broker and controller reports one figure for both of its pools together — up to 200% — so on such a node it is left out rather than shown wrong.

Where each figure shows

• Brokers ▸ tiles — Active controller, Offline partitions, Under min ISR.
• Brokers ▸ list — the optional State and Offline log dirs columns.
• Brokers ▸ Graphs — byte rates, requests per second and their p99 time by type, handler and network idle, queues, messages in per broker.
• A broker's Metrics tab, and a node's figures from the Quorum tab — the JVM and system group, metadata lag, and for a broker its log flush p99, ISR changes, failed requests and rejected bytes.
• A topic's page — its own rates, read when the page is open; never the topics list.
• Alerts — every figure that describes something worth acting on is an alert metric; the per-topic rates are not.
• Supported features — the connection's JMX metrics row.

Brokers and controllers

On a KRaft cluster with separate controllers, each controller that is not a broker is a node too. BROKA asks it at the address you give it in the JMX settings, where it is listed as Controller N after the brokers, or at the host the quorum names and the JMX port. A node that is both broker and controller is listed once, as its broker. A controller has no broker figures and no row in the broker list; its figures open from its row on the Quorum tab.

When not every node answers

A figure about one node is shown for each node that answered. A figure about the whole cluster is shown only when every node it depends on answered: the number of active controllers needs every controller, the under-min-ISR total every broker, the offline-partition count the one node that says it is the active controller. Adding up the nodes that did answer would report a healthy cluster exactly when part of it is not answering. The figures that are not shown say why.

Messages in is never added up across brokers. Each broker counts the messages it writes, and a message replicated to three brokers is counted three times. The figure is per broker; a cluster total is the offset rate the Graphs tab already draws.

Older Kafka, and figures that appear later

A bean a broker does not have is a missing value — never 0, never an error. Measured against both ends of the supported range, Kafka 3.8.0 answers 39 of the 43 figures a node is asked for and Kafka 4.3.1 answers 40: the leader-election rate and time are a ZooKeeper-era controller bean that no KRaft controller registers, unclean elections were answered by 4.3.1's controller only, and follower-fetch requests appear once a broker has followers fetching from it.

What it costs

BROKA reads every node at once and waits three seconds at most for all of them together, so a node that does not answer slows nothing; it is left alone for thirty seconds before it is asked again. Each node's connection is kept open between readings. Measured on a three-node cluster with an authenticated agent on each node, one node's whole list took about 270 milliseconds the first time and about 60 milliseconds after that, moving some 23 KB over the wire; a whole Graphs refresh, the Admin reading included, about 100 milliseconds. Every figure is therefore read on every refresh — every fifteen seconds while the Graphs tab is open, and once a minute for the alert rules, which share the reading.

Without JMX

Everything that does not need JMX works the same. Each place that does says so: a tile reads Needs JMX and opens the JMX settings, a chart or the Metrics tab says where to set it up, a topic shows no rates, and a rule on a JMX figure reads unevaluatable with the reason. The connection's Supported features panel says whether the last reading reached a node.

Nothing here is stored

The figures are read when a screen or a rule asks for them and are never written down. There is no history behind them; a trend is a rule's hold for window, not a chart.