Skip to content

CleanerThread

CleanerThread is a non-daemon thread of execution for LogCleaner for log cleanup (one at a time until no more is left).

Creating Instance

CleanerThread takes the following to be created:

  • Thread ID

CleanerThread is created when:

Thread Name

CleanerThread uses the following as the thread name (with the threadId)

kafka-log-cleaner-thread-[threadId]

log.cleaner.threads

The number of CleanerThreads (that LogCleaner uses) is controlled by log.cleaner.threads dynamic configuration.

doWork

doWork(): Unit

doWork is part of the ShutdownableThread abstraction.

doWork tryCleanFilthiestLog. If no logs was cleaned up, doWork pauses the thread for log.cleaner.backoff.ms millis.

In the end, doWork requests the LogCleanerManager to maintainUncleanablePartitions.

tryCleanFilthiestLog

tryCleanFilthiestLog(): Boolean

tryCleanFilthiestLog cleanFilthiestLog.

LogCleaningException

In case of LogCleaningException, tryCleanFilthiestLog prints out the following WARN message to the logs:

Unexpected exception thrown when cleaning log [log].
Marking its partition ([topicPartition]) as uncleanable

doWork requests the LogCleanerManager to maintainUncleanablePartitions and returns false.

Cleaning Filthiest Log

cleanFilthiestLog(): Boolean

cleanFilthiestLog returns cleaned flag to indicate whether a log to clean was found or not.

cleanFilthiestLog requests the LogCleanerManager to grabFilthiestCompactedLog.

If there is no log to clean up, cleanFilthiestLog "returns" false (as cleaned flag). Otherwise, cleanFilthiestLog cleanLog and "returns" true.

cleanFilthiestLog requests the LogCleanerManager for deletableLogs and then every UnifiedLog to deleteOldSegments.

In the end, cleanFilthiestLog requests the LogCleanerManager to doneDeleting (with the TopicPartitions).

Logging

CleanerThread uses kafka.log.LogCleaner logger.