Skip to content

= [[HistoryServer]] HistoryServer -- WebUI For Active And Completed Spark Applications

HistoryServer is an extension of the[web UI] for reviewing event logs of running (active) and completed Spark applications with event log collection enabled (based on[spark.eventLog.enabled] configuration property).

HistoryServer supports custom[configuration properties].

HistoryServer is <> when...FIXME

HistoryServer uses the <> to handle requests to /* URI that <>.

[[ApplicationCacheOperations]] HistoryServer is a[ApplicationCacheOperations].

[[UIRoot]] HistoryServer is a[UIRoot].

[[retainedApplications]] HistoryServer uses[spark.history.retainedApplications] configuration property (default: 50) for...FIXME

[[maxApplications]] HistoryServer uses[spark.history.ui.maxApplications] configuration property (default: unbounded) for...FIXME

[[logging]] [TIP] ==== Enable ALL logging level for org.apache.spark.deploy.history.HistoryServer logger to see what happens inside.

Add the following line to conf/

Refer to[Logging].

== [[creating-instance]] Creating HistoryServer Instance

HistoryServer takes the following to be created:

  • [[conf]][SparkConf]
  • [[provider]][ApplicationHistoryProvider]
  • [[securityManager]] SecurityManager
  • [[port]] Port number

HistoryServer initializes the <>.

While being created, HistoryServer is requested to <>.

== [[initialize]] Initializing HistoryServer -- initialize Method

[source, scala]

initialize(): Unit

NOTE: initialize is part of[WebUI Contract] to initialize web components.


== [[attachSparkUI]] attachSparkUI Method

[source, scala]

attachSparkUI( appId: String, attemptId: Option[String], ui: SparkUI, completed: Boolean): Unit

NOTE: attachSparkUI is part of[ApplicationCacheOperations Contract] to...FIXME.


== [[main]] Launching HistoryServer Standalone Application -- main Method

[source, scala]

main(argStrings: Array[String]): Unit


== [[getAppUI]] Requesting Spark Application UI -- getAppUI Method

[source, scala]

getAppUI(appId: String, attemptId: Option[String]): Option[LoadedAppUI]

NOTE: getAppUI is part of[ApplicationCacheOperations Contract] to...FIXME.


== [[withSparkUI]] withSparkUI Method

[source, scala]

withSparkUIT(fn: SparkUI => T): T

NOTE: withSparkUI is part of[UIRoot Contract] to...FIXME.


== [[loadAppUi]] loadAppUi Internal Method

[source, scala]

loadAppUi(appId: String, attemptId: Option[String]): Boolean


NOTE: loadAppUi is used exclusively when HistoryServer is <>.

== [[doGet]] doGet Method

[source, scala]

doGet(req: HttpServletRequest, res: HttpServletResponse): Unit

NOTE: doGet is part of Java Servlet's[HttpServlet] to handle HTTP GET requests.


NOTE: doGet is used when...FIXME

== [[internal-properties]] Internal Properties

[cols="30m,70",options="header",width="100%"] |=== | Name | Description

| appCache a| [[appCache]][ApplicationCache] for this HistoryServer and <>

Used when...FIXME

| loaderServlet a| [[loaderServlet]] Java Servlets'[HttpServlet]

Used exclusively when HistoryServer is requested to <> (and[attaches the servlet to the web UI] to handle /* URI)


Last update: 2020-10-06