Skip to content

LogCleaner

LogCleaner is created and started immediately for LogManager when started with log.cleaner.enable configuration property enabled.

Creating Instance

LogCleaner takes the following to be created:

  • CleanerConfig
  • Log directories
  • UnifiedLogs by TopicPartition (Pool[TopicPartition, UnifiedLog])
  • LogDirFailureChannel
  • Time

LogCleaner is created when:

LogCleanerManager

LogCleaner creates a LogCleanerManager (with the log directories and UnifiedLogs) when created.

KafkaMetricsGroup

LogCleaner is a KafkaMetricsGroup.

BrokerReconfigurable

LogCleaner is a BrokerReconfigurable.

Starting Up

startup(): Unit

startup prints out the following INFO message to the logs:

Starting the log cleaner

startup creates log.cleaner.threads of CleanerThreads that are started immediately.

startup adds the CleanerThreads to the cleaners registry.


startup is used when:

Creating CleanerConfig

cleanerConfig(
  config: KafkaConfig): CleanerConfig

cleanerConfig creates a CleanerConfig with the configuration properties from the given KafkaConfig.


cleanerConfig is used when:

CleanerThreads

LogCleaner uses CleanerThreads for log cleaning.

LogCleaner creates log.cleaner.threads of CleanerThreads when started up. They are shut down when shutdown.

Use the following metrics to monitor the threads:

Logging

Enable ALL logging level for kafka.log.LogCleaner logger to see what happens inside.

Add the following line to config/log4j.properties:

log4j.logger.kafka.log.LogCleaner=ALL

Refer to Logging.

Note

Kafka comes with a preconfigured kafka.log.LogCleaner logger in config/log4j.properties:

log4j.appender.cleanerAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.cleanerAppender.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.cleanerAppender.File=${kafka.logs.dir}/log-cleaner.log
log4j.appender.cleanerAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.cleanerAppender.layout.ConversionPattern=[%d] %p %m (%c)%n

log4j.logger.kafka.log.LogCleaner=INFO, cleanerAppender
log4j.additivity.kafka.log.LogCleaner=false

That means that the logs go to logs/log-cleaner.log file at INFO logging level and are not added to the main logs (per log4j.additivity).