Kafka Interview Questions & Answers (2026)
These interviews test your grasp of Kafka core concepts, cluster management, and real‑time streaming design. Demonstrate clear knowledge of topics like partitions, replication, consumer groups, and exactly‑once semantics. Show practical experience with configuration trade‑offs, monitoring, and integration patterns. To succeed, explain why choices matter, cite concrete scenarios, and articulate how you’d diagnose and resolve common issues.
22 questions · updated Aug 29, 2026
Quick facts
| Typical rounds | Phone screen, technical deep‑dive, system design, and on‑site coding |
| Core topics | Producers, brokers, consumers, replication, partitioning, and stream processing |
| Preferred experience | 2‑5 years with Kafka in production, familiarity with Zookeeper or KRaft |
| Key metrics | Throughput (MB/s), latency (ms), ISR size, and consumer lag |
| Common tools | Kafka‑Connect, ksqlDB, Confluent Control Center, Prometheus |
Questions
Beginner
What is a Kafka partition and why is it important?
A partition is an ordered, immutable sequence of records within a topic, stored on a broker. It enables parallelism because each consumer in a group can read from a distinct partition, scaling throughput. The ordering guarantee is per‑partition, not across partitions, so choosing the right partition key balances load while preserving needed order. Interviewers look for understanding of both scalability and ordering implications.
How does Kafka achieve fault tolerance?
Kafka replicates each partition across multiple brokers defined by the replication factor. The leader handles all reads and writes, while followers copy the log. If the leader fails, one in‑sync replica (ISR) is elected as the new leader, ensuring no data loss as long as the min.insync.replicas condition is met. Candidates should mention the role of Zookeeper/KRaft in controller elections and the trade‑off between durability and latency.
Explain the difference between at‑least‑once and exactly‑once delivery semantics.
At‑least‑once guarantees that every record is delivered, but duplicates may appear if retries occur. Exactly‑once ensures each record is processed only once, using idempotent producers and transactional writes that atomically commit offsets with the produce operation. Interviewers expect you to discuss the performance cost of transactions and when exactly‑once is justified, such as financial or inventory systems.
What is the role of the consumer group offset?
The offset tracks the last committed position a consumer group has read in each partition. Stored in the __consumer_offsets topic, it enables fault‑tolerant restarts and load balancing. A candidate should explain how committing offsets manually vs. automatically affects processing guarantees, and how lag monitoring uses these offsets to detect bottlenecks.
Why does Kafka prefer sequential disk writes?
Kafka writes logs sequentially to take advantage of OS page cache and SSD write‑amplification characteristics, achieving high throughput and low latency. Random reads are still fast because each segment is memory‑mapped. Interviewers look for an understanding of how this design differs from traditional databases and how it influences retention and compaction strategies.
What is log compaction and when would you use it?
Log compaction retains only the latest record for each key, discarding older updates. It is useful for change‑data‑capture or state reconstruction where you need the current value, not the full history. Candidates should note that compaction runs in the background, does not affect retention time, and requires a key to be defined in the topic.
Intermediate
How does the producer’s acks setting affect durability and latency?
acks=0 sends without waiting for a broker response, giving lowest latency but no durability. acks=1 waits for the leader’s acknowledgment, balancing latency and risk of data loss if the leader crashes before replication. acks=all (or -1) waits for all in‑sync replicas, providing strongest durability at higher latency. Interviewers expect you to discuss the trade‑off and typical production choices.
What is the purpose of the ISR (in‑sync replica) list?
ISR contains replicas that are fully caught up to the leader’s log. Only replicas in ISR are eligible to become the new leader during a failover, ensuring no data loss. The min.insync.replicas broker config defines how many ISR members must acknowledge a write for it to be considered successful, tying directly to durability guarantees.
Describe how Kafka handles back‑pressure from slow consumers.
Kafka decouples producers and consumers via the log; slow consumers simply lag behind, increasing consumer lag metrics. The broker does not block producers unless the log segment reaches its retention limit. Candidates should mention configuring retention.bytes or time, using pause()/resume() in the consumer API, and monitoring lag to trigger scaling or rebalancing.
When would you choose a custom partitioner over the default hash partitioner?
A custom partitioner is useful when you need domain‑specific routing, such as sending related events to the same partition based on business rules, or when you want to balance load unevenly (e.g., hot keys). Explain that the default uses murmur2 hashing, which may cause skew if keys are not uniformly distributed, and how a custom implementation can mitigate that.
Explain the impact of increasing the replication factor on throughput and storage.
Higher replication improves fault tolerance but adds write latency because the leader must wait for acknowledgments from more replicas (depending on acks setting). It also multiplies storage requirements proportionally. Interviewers expect you to discuss the balance between durability, latency, and cost, and how to monitor ISR size to avoid unnecessary replication.
What are the differences between Zookeeper‑based and KRaft (Kafka Raft) controllers?
Zookeeper stores metadata and handles leader election, requiring an external service and adding operational complexity. KRaft embeds Raft consensus directly in Kafka brokers, eliminating Zookeeper, simplifying deployment, and reducing latency for metadata updates. Candidates should note that KRaft is newer, still maturing, and may affect upgrade paths, but offers tighter consistency guarantees.
How does Kafka Streams achieve stateful processing?
Kafka Streams uses local RocksDB stores to maintain state per task, backed by changelog topics that replicate state changes for fault tolerance. When a task fails, the state is rebuilt from the changelog. Interviewers look for understanding of how this enables exactly‑once processing, windowed joins, and the trade‑off of local storage versus network overhead.
What is the effect of setting a high max.poll.records value?
A high max.poll.records allows the consumer to fetch many records per poll, reducing network overhead but increasing processing latency and memory usage. If processing cannot keep up, the consumer may exceed max.poll.interval.ms, causing a rebalance. Candidates should discuss balancing batch size with processing time and how to tune these settings for throughput‑critical workloads.
Advanced
Explain how Kafka’s exactly‑once semantics work with transactions.
Kafka transactions group producer writes and offset commits into an atomic unit. The producer obtains a transactional ID, begins a transaction, writes records, and then sends a commit marker. Consumers that read within the same transaction see either all or none of the records, preventing duplicates. Interviewers expect you to mention idempotent producers, the need for enable.idempotence, and the performance impact of transaction logs.
How would you design a system to guarantee low latency (<10 ms) for 100 k messages per second?
Use a dedicated high‑throughput topic with a small number of partitions matching the number of producer threads, enable compression=none, acks=all with min.insync.replicas=2, and place brokers on SSDs with ample network bandwidth. Co‑locate producers and brokers in the same data center, tune socket.send.buffer.bytes, and use the Java producer’s batch.size and linger.ms to batch minimally. Monitoring end‑to‑end latency with Prometheus and adjusting replication lag is essential.
What are the challenges of scaling consumer groups to thousands of members?
When a group has many members, partition assignment becomes a bottleneck, and rebalancing overhead grows, causing increased latency. Coordination via the group coordinator can saturate, and each member maintains its own offset state, increasing load on the __consumer_offsets topic. Solutions include increasing partition count, using static membership (group.instance.id), and limiting group size by sharding workloads across multiple groups.
Describe the impact of using tiered storage on Kafka performance.
Tiered storage offloads older segments to remote object storage, reducing local disk usage and enabling longer retention without adding hardware. Reads of recent data remain fast, but fetching older data incurs network latency. Interviewers expect you to discuss configuration of remote.storage.enable, the trade‑off between cost and latency, and how compaction interacts with tiered storage.
How do you troubleshoot a sudden increase in consumer lag?
First, check broker health (CPU, disk I/O, network) and ISR size. Verify that the consumer’s processing time per batch has increased; profile code or adjust max.poll.records. Examine topic retention settings to ensure logs aren’t being truncated prematurely. Use kafka-consumer-groups.sh to view lag per partition, and consider scaling out the consumer group or rebalancing partitions.
Explain the role of the leader epoch in Kafka’s replication protocol.
Leader epoch increments each time a new leader is elected for a partition. It is stored in the log and used by followers to detect stale replicas and avoid writing out‑of‑order data. During fetch requests, the broker includes the current epoch, allowing the consumer to detect a leader change and handle potential duplicates. Interviewers look for awareness of epoch handling in client libraries.
What are the security implications of enabling SASL Plain vs. SCRAM‑SHA‑256?
SASL Plain transmits credentials in clear text and should only be used over TLS, while SCRAM‑SHA‑256 hashes passwords, providing stronger protection against credential leakage. SCRAM also supports password rotation and server‑side verification without storing plain passwords. Candidates should discuss the need for encryption in transit, the impact on authentication latency, and compliance considerations.
How would you implement exactly‑once processing across multiple Kafka topics?
Use a single transactional producer with a unique transactional ID to write to all target topics within one transaction, then commit offsets for the source topic in the same transaction. This ensures that either all writes and the offset commit succeed together or none do, preserving exactly‑once semantics across topics. Explain the need for enable.idempotence, proper isolation.level=read_committed for consumers, and handling of aborts.
Common mistakes
- Confusing partitions with topics and assuming global ordering
- Setting acks=all without considering latency impact
- Ignoring consumer lag and assuming Kafka auto‑scales
- Using default compression for latency‑critical pipelines
- Misconfiguring min.insync.replicas leading to data loss
Study plan
- Review core concepts: topics, partitions, replication, ISR, and offsets.
- Hands‑on: set up a local cluster, produce/consume with different acks and compression settings.
- Deep dive: transactions, exactly‑once semantics, and Kafka Streams state stores.
- Practice troubleshooting: simulate broker failures, consumer lag, and rebalancing.
- Mock interview: answer 18 questions aloud, focusing on trade‑offs and real‑world examples.
FAQ
Do I need Zookeeper to run Kafka in 2026?
Kafka 3.0 introduced KRaft, which removes the Zookeeper dependency. New deployments can use KRaft for metadata management, but many enterprises still run Zookeeper for backward compatibility. Choose based on your organization’s upgrade path and feature requirements.
What is the best way to monitor Kafka performance?
Export JMX metrics to Prometheus or Grafana, track broker CPU, disk I/O, network, ISR size, and consumer lag. Use Confluent Control Center or open‑source tools like Kafka‑Lag‑Exporter for real‑time alerts. Correlate metrics with application latency to pinpoint bottlenecks.
How does log compaction differ from retention policies?
Retention policies delete records based on time or size, removing all data after the threshold. Log compaction retains the latest record for each key regardless of age, allowing you to keep the current state while discarding older updates. Both can be applied simultaneously on the same topic.
Can Kafka guarantee message ordering across partitions?
Kafka guarantees ordering only within a single partition. To preserve order for related events, you must route them to the same partition using a consistent key. Cross‑partition ordering requires additional coordination, such as using a stream processor that re‑orders events after consumption.
When should I use Kafka Connect vs. custom producers?
Kafka Connect is ideal for moving large volumes of data between Kafka and external systems with minimal code, using pre‑built connectors. Custom producers are better for fine‑grained control, complex transformations, or when you need low‑latency, application‑specific logic not covered by existing connectors.
Related
Ready for your next interview?
Download MiPrep AI. Load your resume and the job description. Show up ready.
Free tier · No credit card · macOS 14+ · Windows 10+
Free tier · No credit card · Runs on your Mac or Windows machine