FsHistoryProvider — File-System-Based History Provider

FsHistoryProvider is the default ApplicationHistoryProvider for Spark History Server.

FsHistoryProvider is created exclusively when HistoryServer is started as a standalone application and spark.history.provider configuration property was not defined.

Enable INFO or DEBUG logging levels for org.apache.spark.deploy.history.FsHistoryProvider logger to see what happens inside.

Add the following line to conf/log4j.properties:


Refer to Logging.

rebuildAppStore Internal Method

  store: KVStore,
  eventLog: FileStatus,
  lastUpdated: Long): Unit


rebuildAppStore is used when…​FIXME

getAppUI Method

getAppUI(appId: String, attemptId: Option[String]): Option[LoadedAppUI]
getAppUI is part of ApplicationHistoryProvider Contract to…​FIXME.


Creating FsHistoryProvider Instance

FsHistoryProvider takes the following when created:

FsHistoryProvider initializes the internal registries and counters.