Log Cleanup¶

Kafka uses cleanup.policy configuration property to apply cleanup strategies (policy) to logs:

• Log Compaction
• Log Retention

Configuration Properties¶

The cluster-wide log.cleanup.policy and the per-topic cleanup.policy configuration properties are comma-separated lists of cleanup strategies:

• compact - enables log compaction
• delete - enables log retention

Unless defined, cleanup.policy is exactly log.cleanup.policy.

Log Compaction¶

Log Compaction is a cleanup strategy in which...FIXME

Kafka brokers use LogCleaner for compact retention strategy.

Log compaction can be reconfigured dynamically at runtime.

Log Retention¶

Log Retention (Garbage Collection) is a cleanup strategy to discard (delete) old log segments when their retention time or size limit has been reached.

By default there is only a time limit and no size limit.

Kafka brokers schedule kafka-log-retention periodic task for delete retention strategy.

Kafka uses log.retention.check.interval.ms configuration property as the interval between regular log checks.

Time-Based Retention¶

Retention Time is controlled by the cluster-wide log.retention.ms, log.retention.minutes or log.retention.hours configuration properties (from the highest to the lowest priority) or their per-topic retention.ms configuration property.

Size-Based Retention¶

Retention Size is controlled by the cluster-wide log.retention.bytes or per-topic retention.bytes configuration property.

Logging¶

Enable ALL logging level for kafka.log.Log logger to see messages related to log retention.