BROKABROKA
Sign inDownload CommunityRequest a demo
PlatformKafka destinations shipped

Change data flows, under the same operations layer.

The operational model BROKA already applies to Kafka, Redis, RabbitMQ and Apache Artemis (formerly ActiveMQ Artemis) extends naturally to the databases those messages come from. We are exploring change data capture as the next technology in the same console: connect a SQL Server or PostgreSQL database, see whether it is ready for CDC, turn capture on deliberately, inspect what is actually being captured, and hand your Kafka Connect cluster a connector configuration that matches.

Status
  • Kafka, Redis, RabbitMQ, ArtemisAvailable now
  • SQL Server, PostgreSQL and MySQL to KafkaAvailable now
  • Non-Kafka destinationsPlanned
  • Connector operationsAvailable now

This page describes intent, not a delivery commitment. No dates, versions or compatibility guarantees are implied.

01The shape of the flow
BROKA — OPERATIONS AND GOVERNANCE LAYERreadiness · enable · inspect · generate config · connector health · offsets · access control · audit · end-to-end investigationsource databaseSQL Server · CDC tablesPostgreSQL · publication+ replication slotchangesDebezium connectoron your Kafka Connectcluster — registered withBROKA over its APIsnapshot → streamingdestinationKafka topic per tableRedis · RabbitMQplanned, Lane Bconsumerapplications andconsumer groupsretry · dead-letter · error stateChange data moves through your database, your Connect cluster and your brokers. BROKA observes and manages it — it is not in the data path and stores no change data.
02What we are exploring

Six capabilities, one operating habit.

The point is not another CDC runtime. It is the operational work around it — knowing whether a database is ready, turning capture on without guesswork, seeing what is captured, and handing the runtime a configuration that matches what you just enabled.

01

Direct database sources

SQL Server and PostgreSQL connect as sources beside your broker connections — per environment, with credentials held and encrypted the same way.

  • engine · host · database
  • TLS and credential handling
  • connection test with reasons
02

CDC readiness check

Before anything is changed, a read-only pass over the things that decide whether capture will work at all — with what to fix, not just a red cross.

  • sql server · cdc enabled, agent jobs, retention
  • postgresql · wal_level, slots, publications
  • privileges the login actually has
03

Deliberate enabling

Pick tables, see the statements BROKA intends to run, then either run them under the environment policy or take them away to be reviewed.

  • preview before write
  • guarded by role and environment
  • recorded in the audit timeline
04

Captured table inspection

What is being captured, since when, with which key and which columns — and whether changes are actually arriving or the table has been quiet for a day.

  • capture instance · publication · identity
  • change volume and latest position
  • before and after images, read-only
05

Connector configuration

Generated from the tables you enabled and the destination you chose. The destination also decides where it runs, so there is no runtime to pick: a Kafka destination produces a Connect REST payload, and a Redis or RabbitMQ destination produces a pipeline BROKA runs itself.

  • connect json · or a pipeline definition
  • topic, stream and routing naming
  • what this will create, in words
06

Connector operations

Snapshot or streaming, position against the source, errors and retries — with pause, resume and restart as guarded operations.

  • task state and failures
  • offsets against the source
  • pause, resume and restart
03Enabling capture

Run it, or take the statements to your DBA.

Enabling capture is a schema-level change, so it is never silent. BROKA shows exactly what it intends to do, then offers two paths — and in an environment marked read-only, only the second one is available.

Path ANeeds operate rights

BROKA runs the statements

For development, test and staging, where the console is allowed to alter the source schema. Every statement is shown first and recorded afterwards.

  • —Confirmation names the tables and the database
  • —Rolled up into one audited action
  • —Readiness re-checked immediately after
Path BDefault in production

BROKA generates, someone else runs

For estates where a console may not touch a production schema. The generated SQL goes into a change request, a migration or a DBA ticket unchanged.

  • —Copy or download the statements
  • —No write attempted against the database
  • —BROKA verifies the result once it has been applied
Generated for the tables you selected
Illustrative — the real statements are read from your database
enable-cdc.sqlserver.sql
-- illustrative: BROKA reads the current state and writes what is missing
-- 1 · database level, once
EXEC sys.sp_cdc_enable_db;

-- 2 · per table selected in the console
EXEC sys.sp_cdc_enable_table
     @source_schema = N'sales',
     @source_name   = N'orders',
     @role_name     = N'cdc_reader',
     @capture_instance = N'sales_orders',
     @supports_net_changes = 1;

-- 3 · what BROKA will verify afterwards
--    capture instance created, agent jobs present,
--    retention window, and the columns actually captured
04Runtime and destinations
Lane A · shippedMany connectors per cluster

Kafka destinations, on Kafka Connect

When the destination is a Kafka topic, capture runs as a Debezium connector on your Connect cluster and is operated from BROKA beside your topics and consumer groups. This path is finished: SQL Server, PostgreSQL and MySQL sources are proven against live databases, including schema evolution, dead-letter triage and day-two repair from inside the console.

  • —Connector and task state, offsets, pause, resume and restart
  • —Schema history and offsets handled by the cluster
  • —One cluster carries many capture connectors
BROKA generates · connector JSON for the Connect REST API
Lane B · plannedOne process, many pipelines

Redis and RabbitMQ destinations, on a worker BROKA runs

Kafka Connect always terminates in Kafka, so a destination that is not Kafka needs a different engine. BROKA embeds Debezium Engine in a worker of its own rather than attaching to a third-party runtime — which is what makes the sink ours, and therefore what makes retry and dead-lettering possible at all.

  • —Sinks BROKA writes: a Redis stream or a cache projection, a RabbitMQ exchange and routing key
  • —Offsets and schema history in the PostgreSQL you already run
  • —Start, stop, pause, resume and snapshot control as real operations
BROKA generates · a pipeline definition it runs itself

Whoever runs it, it runs in your infrastructure.

In Lane A the runtime is yours: your team stands up Kafka Connect and registers it with BROKA over its API, the same way a broker connection is added. Lane B is planned so that there is nothing for you to stand up — BROKA would run the worker itself, as a second process in your own deployment. Either way the change data never leaves your infrastructure, and connectors, offsets and failures appear beside the rest of your estate.

  • 01You deploy Connect and Debezium where you want them to run
  • 02You register the cluster in BROKA over its API, per environment
  • 03BROKA builds the connector configuration with you, checked by the Connect cluster’s own validation
  • 04Connector state, position and errors are operated like any other resource
Destinations
  • Kafka topicsDebezium on Kafka Connect — the primary pathPreferred
  • RabbitMQ streams and exchangesAn exchange and routing-key template, with an explicit policy for deletesLane B · planned
  • Redis StreamsEither an append-only stream or a cache projection — keys written and deleted to match the sourceLane B · planned
  • Artemis destinationsNot a destination in either lane. The worker's sinks are Redis and RabbitMQ, and we would rather say so than promise a bridge nobody has builtOpen question

Kafka destinations are finished and proven against live databases. Redis and RabbitMQ are planned, on a worker BROKA would run itself rather than a runtime it attaches to; none of it is built yet. Artemis is in neither lane, and saying so is better than promising a bridge nobody has built.

Boundary

In Lane A, BROKA manages and never carries: it reads connector state and offsets, and the delivery path is your Connect cluster. Lane B, as designed, carries as well — the worker would be what reads the source and writes the sink — so it would be in the delivery path by design, and says so rather than borrowing Lane A's boundary. Change data is never stored by BROKA and never leaves your infrastructure in either lane.

Lane B is designed for at-least-once delivery. Exactly-once is out of scope, so a consumer downstream of a pipeline would have to be idempotent — duplicates would be possible after a restart or a failover, and no change would be lost. Concurrency is to be guarded per pipeline: a second worker could never double-run one, and a crashed worker's pipelines would be picked up.

05What stays the same

A database source is a connection. A captured table is a resource. Enabling capture is a guarded write. None of that is new machinery — it is the model already running under every broker BROKA supports, applied to one more technology.

EnvironmentsConnectionsResourcesHealthEventsAccess controlAuditGuardrailsSearchTroubleshooting
06Platform extensibilityProduct direction

Designed to expand beyond today's broker platforms.

Kafka, Redis, RabbitMQ and Apache Artemis are fully supported today. The operational model underneath them — environments, connections, resources, health, access control, audit, guardrails and search — is built as a foundation for messaging and data movement more broadly, so new technologies can join it without a new tool.

Product categories
  • Message brokersAvailable now
  • Streaming platformsProduct direction
  • Change data captureAvailable now
  • Data integrationProduct direction
  • Connector operationsAvailable now
  • Event-flow governanceProduct direction

Whatever joins the platform inherits the same operational foundation — and keeps its own specialized concepts and operations.

Launch scope is the brokers supported today. Direction items carry no dates or delivery commitments.

Shape this with us

Tell us how your change flows actually run.

If you are moving change data out of SQL Server or PostgreSQL today, the details of how you do it are what we want to hear — engines, destinations, who is allowed to alter a schema, and what breaks at three in the morning.

Talk to us about CDCSee what ships todayDocumentation →