Skip to content

Transactions

Apache Kafka supports transactional record delivery (and consumption if in consumer-process-produce processing mode).

Every Kafka broker runs a TransactionCoordinator to manage (coordinate) transactions.

Transactional Producer

A KafkaProducer is transactional when transactional.id configuration property is specified.

Any record sending has to be after KafkaProducer.initTransactions followed by KafkaProducer.beginTransaction. Otherwise, the underlying TransactionManager is going to be in a wrong state (that will inevitably lead to exceptions).

Transaction-Aware Consumer

A KafkaConsumer supports transactions using isolation.level configuration property.

kafka-console-consumer

kafka-console-consumer supports --isolation-level option for isolation.level configuration property.

Demo

Demo: Transactional Kafka Producer

Transactional Configuration Properties

TransactionConfig

Transaction Topic

Kafka brokers use __transaction_state internal topic for managing transactions (as records).

__transaction_state is auto-created at the first transaction.

The number of partitions is configured using transaction.state.log.num.partitions configuration property.

A transaction (record) is assigned a partition (txn topic partition) based on the absolute hash code of the transactional.id.

Learning Resources