KRaft

The Controller Quorum That Killed ZooKeeper

KRaft (Kafka Raft) replaces the ZooKeeper-based control plane with a Raft consensus group running inside Kafka itself. The cluster's metadata — topics, partitions, configurations, ACLs, broker registrations — lives in a single internal log, the __cluster_metadata topic, replicated across the controller quorum. Brokers fetch from this log to learn the cluster state.

The architectural simplification is significant: one Raft log replaces ZooKeeper's znodes, watches, and ZAB protocol. Failover that took minutes on large clusters now takes seconds. Production at scale (millions of partitions) becomes feasible. KRaft became the default in Kafka 3.3, and ZooKeeper support was removed in Kafka 4.0.

Architecture

Three controllers form a Raft quorum. Brokers register as observers and fetch metadata.

Key Numbers

3-5

Recommended controllers per cluster

~10x

Faster controller failover vs ZK

2M+

Partitions per cluster (vs ~200K with ZK)

3.3

First production-ready KRaft release

4.0

Kafka version that removed ZooKeeper

N/2+1

Quorum size for writes

~1s

Typical metadata propagation lag

The __cluster_metadata Topic

A single internal topic with one partition, replicated across the controller quorum. Each record is a metadata change.

# Inspect the metadata log directly
$ kafka-metadata-shell.sh --snapshot /var/lib/kafka/__cluster_metadata-0/00000000000000000000.log

>> ls /
brokers/
topics/
configs/
acls/

>> ls /topics
orders
payments
__consumer_offsets

>> cat /topics/orders/0/data
{
  "partitionId": 0,
  "topicId": "Pl3Y3...uuid",
  "replicas": [1, 2, 3],
  "isr": [1, 2, 3],
  "leader": 1,
  "leaderEpoch": 17,
  "partitionEpoch": 23
}

The log is periodically snapshotted so new controllers do not need to replay from offset 0. Snapshot is taken every metadata.log.max.snapshot.interval.ms (default 1 hour) and after a configurable number of bytes. Old log segments before the snapshot can be deleted.

Configuring a KRaft Cluster

# controller.properties (3 voters)
process.roles=controller
node.id=1
controller.quorum.voters=1@kafka-c1:9093,2@kafka-c2:9093,3@kafka-c3:9093
listeners=CONTROLLER://kafka-c1:9093
controller.listener.names=CONTROLLER
log.dirs=/var/lib/kafka/metadata

# broker.properties (data nodes)
process.roles=broker
node.id=101
controller.quorum.voters=1@kafka-c1:9093,2@kafka-c2:9093,3@kafka-c3:9093
listeners=PLAINTEXT://kafka-b1:9092
log.dirs=/var/lib/kafka/data

# Combined mode (small clusters / dev)
process.roles=broker,controller

# Format the storage directory before first start
$ KAFKA_CLUSTER_ID=$(bin/kafka-storage.sh random-uuid)
$ bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c controller.properties

The cluster ID is now a UUID generated at format time, not assigned by ZooKeeper. All controllers and brokers must format with the same cluster ID. The format step writes a meta.properties file in log.dirs that pins the node to that cluster.

Why KRaft Is Faster

Operation	ZooKeeper	KRaft
Topic creation	Multiple ZK round-trips, watch fires on every broker	One log append, brokers pick up via fetch
Controller failover	New controller reads all znodes (minutes for large clusters)	New leader already has the log; promotes in seconds
Broker registration	Ephemeral znode + watch propagation	RegisterBrokerRecord appended to log
Partition reassignment	ZK znode + watch + state machine in controller	PartitionChangeRecord in metadata log
Metadata read by broker	UpdateMetadataRequest pushed by controller	Broker fetches from log, replays in order

The architectural insight: by making metadata changes just another log, Kafka reuses the same well-tested replication machinery that powers data partitions. There is no separate state machine to reason about, no watch/notification fan-out, no two-source-of-truth problem.

Migration from ZooKeeper

KIP-866 defines a phased migration. The cluster runs in dual-write mode until all brokers are upgraded.

# Phase 1: bring up KRaft controllers in migration mode
zookeeper.metadata.migration.enable=true
zookeeper.connect=zk-1:2181,zk-2:2181,zk-3:2181
controller.quorum.voters=1@kc1:9093,2@kc2:9093,3@kc3:9093

# Phase 2: rolling restart brokers with the new config
process.roles=broker
zookeeper.metadata.migration.enable=true
zookeeper.connect=zk-1:2181,...
controller.quorum.voters=1@kc1:9093,...

# Phase 3: monitor migration progress
$ kafka-features.sh --bootstrap-server kafka-b1:9092 describe
metadata.version: 3.5-IV2  (migration in progress)

# Phase 4: cut over — disable ZK on controllers
# remove zookeeper.metadata.migration.enable and zookeeper.connect

# Phase 5: rolling restart brokers with KRaft-only config
# delete the ZK ensemble

Tradeoffs and When Not to Use

Use KRaft when

Any new Kafka deployment (it is the only option in 4.0+)
You have many partitions or fast topic creation
You want one fewer system to operate
You need fast controller failover

Caveats

Older client libraries may have edge-case bugs with the new metadata RPCs
Operational tooling rebuilt: kafka-reassign-partitions, kafka-acls, kafka-configs all use new APIs
Migration of large running clusters requires careful testing
Some older Confluent components hardcoded ZooKeeper paths

Frequently Asked Questions

Why was ZooKeeper a bottleneck?

Every metadata change (topic creation, partition reassignment, ACL update) had to round-trip through ZooKeeper, which is itself a Raft-like ensemble. The controller broker watched ZooKeeper for changes and propagated them to the cluster, but ZooKeeper's write throughput is modest (a few thousand ops/sec) and its watch model produces thundering herds during failovers. Large clusters with millions of partitions stalled for minutes during controller failover because the new controller had to read all metadata from ZooKeeper before becoming useful.

How does the controller quorum elect a leader?

The KRaft controllers run a Raft consensus protocol over the __cluster_metadata topic. One controller is the active leader, the others are followers. Leader election uses Raft's standard term/vote mechanism: if the leader misses heartbeats, a follower bumps the term and requests votes. Whichever candidate gets a majority becomes the new leader. Crucially, the metadata log itself elects the leader — there is no separate election system.

Can I run a mixed ZooKeeper + KRaft cluster during migration?

Yes. Kafka 3.4+ supports a migration mode where the new KRaft quorum runs alongside the existing ZooKeeper ensemble. The controller is the source of truth, but it dual-writes to ZooKeeper so old brokers can still read. Brokers are upgraded one at a time to KRaft mode. After all brokers migrate, you flip the cluster to KRaft-only and delete ZooKeeper. This took several Kafka releases to stabilize and is documented in KIP-866.

How many controllers should I run?

Three for production (tolerates 1 failure), five for very large clusters (tolerates 2 failures). Controllers should be on dedicated nodes — they can be co-located with brokers in small deployments, but in production you separate them so a broker GC pause does not affect controller liveness. The metadata log is small (megabytes typically), so controllers do not need much disk.

How is metadata propagated to brokers?

Brokers fetch from the metadata log just like consumers fetch from a normal topic — using the standard FetchRequest. The broker maintains a local copy of the metadata log and applies records in order to update its in-memory metadata cache. This means metadata propagation has the same well-understood semantics as data replication, and brokers can be many records behind without affecting cluster correctness (only their view of the world).