What to do when there is no initial offset in Kafka or if the current offset does not exist anymore on a broker (e.g. because that data has been deleted)
latest- reset the offset to the latest offset
earliest- reset the offset to the earliest offset
none- throw exception to the consumer if no previous offset is found for the consumer's group
A unique identifier of the consumer group a consumer belongs to. Required if the consumer uses either the group management functionality by using Consumer.subscribe or the Kafka-based offset management strategy.
Controls how KafkaConsumer should read messages written transactionally
read_uncommitted- Consumer.poll() will only return transactional messages which have been committed (filtering out transactional messages which are not committed).
read_committed- Consumer.poll() will return all messages, even transactional messages which have been not committed yet or even aborted.
Non-transactional messages will be returned unconditionally in either mode.
Messages will always be returned in offset order. Hence, in
consumer.poll() will only return messages up to the last stable offset (LSO), which is the one less than the offset of the first open transaction.
In particular any messages appearing after messages belonging to ongoing transactions will be withheld until the relevant transaction has been completed.
As a result,
read_committed consumers will not be able to read up to the high watermark when there are in-flight transactions.
Further, when in
seekToEnd method will return the last stable offset.
kafka-console-consumer supports setting the property using