ApplicationHistoryProvider

ApplicationHistoryProvider is the base of the history providers of Spark applications.

package org.apache.spark.deploy.history

abstract class ApplicationHistoryProvider {
  // only required methods that have no implementation
  // the others follow
  def getListing(): Iterator[ApplicationInfo]
  def getAppUI(appId: String, attemptId: Option[String]): Option[LoadedAppUI]
  def writeEventLogs(appId: String, attemptId: Option[String], zipStream: ZipOutputStream): Unit
  def getApplicationInfo(appId: String): Option[ApplicationInfo]
}
ApplicationHistoryProvider is a private[history] contract.
Table 1. (Subset of) ApplicationHistoryProvider Contract
Method Description

getListing

getAppUI

SparkUI (the UI of a Spark application)

Used exclusively when HistoryServer is requested for the UI of a Spark application

writeEventLogs

Writes events to a stream

getApplicationInfo

ApplicationHistoryProvider is a Scala abstract class and cannot be created directly, but only as one of the implementations.

FsHistoryProvider is the one and only known implementation of ApplicationHistoryProvider contract in Apache Spark.