MapOutputTrackerWorker — MapOutputTracker for Executors

A MapOutputTrackerWorker is the MapOutputTracker for executors.

MapOutputTrackerWorker uses Java’s thread-safe java.util.concurrent.ConcurrentHashMap for mapStatuses internal cache and any lookup cache miss triggers a fetch from the driver’s MapOutputTrackerMaster.

The only difference between MapOutputTrackerWorker and the base abstract class MapOutputTracker is that the mapStatuses internal registry is an instance of the thread-safe java.util.concurrent.ConcurrentHashMap.

Enable INFO or DEBUG logging level for org.apache.spark.MapOutputTrackerWorker logger to see what happens in MapOutputTrackerWorker.

Add the following line to conf/log4j.properties:

log4j.logger.org.apache.spark.MapOutputTrackerWorker=DEBUG

Refer to Logging.