Skip to content

Log Cleanup

Kafka uses cleanup.policy configuration property to apply cleanup strategies (policy) to logs:

Configuration Properties

The cluster-wide log.cleanup.policy and the per-topic cleanup.policy configuration properties are comma-separated lists of cleanup strategies:

Unless defined, cleanup.policy is exactly log.cleanup.policy.

Log Compaction

Log Compaction is a cleanup strategy in which...FIXME

Kafka brokers use LogCleaner for compact retention strategy.

Log compaction can be reconfigured dynamically at runtime.

Log Retention

Log Retention (Garbage Collection) is a cleanup strategy to discard (delete) old log segments when their retention time or size limit has been reached.

By default there is only a time limit and no size limit.

Kafka brokers schedule kafka-log-retention periodic task for delete retention strategy.

Kafka uses log.retention.check.interval.ms configuration property as the interval between regular log checks.

Time-Based Retention

Retention Time is controlled by the cluster-wide log.retention.ms, log.retention.minutes or log.retention.hours configuration properties (from the highest to the lowest priority) or their per-topic retention.ms configuration property.

Size-Based Retention

Retention Size is controlled by the cluster-wide log.retention.bytes or per-topic retention.bytes configuration property.

Logging

Enable ALL logging level for kafka.log.Log logger to see messages related to log retention.