Skip to content


AppStatusStore in Spark Application

AppStatusStore is available as[SparkContext.statusStore] to other Spark services.

Creating Instance

AppStatusStore takes the following to be created:

AppStatusStore is created when:

  • SparkContext is created (that triggers <>)

  • FsHistoryProvider is requested to[create a LoadedAppUI]

== [[streamBlocksList]] streamBlocksList Method

[source, scala]

streamBlocksList(): Seq[StreamBlockData]


NOTE: streamBlocksList is used when...FIXME

== [[activeStages]] activeStages Method

[source, scala]

activeStages(): Seq[v1.StageData]


NOTE: activeStages is used when...FIXME

Creating Event Store

  conf: SparkConf): AppStatusStore

createLiveStore creates a fully-initialized AppStatusStore.

Internally, createLiveStore creates a[] (with a new[] and the input[SparkConf]).

createLiveStore creates a AppStatusListener (with the ElementTrackingStore created, the input SparkConf and the live flag enabled).

In the end, createLiveStore creates an <> (with the ElementTrackingStore and AppStatusListener just created).

createLiveStore is used when SparkContext is created.

== [[close]] Closing AppStatusStore

[source, scala]

close(): Unit

close simply requests <> to[close].

NOTE: close is used when...FIXME

== [[rddList]] rddList Method

[source, scala]

rddList(cachedOnly: Boolean = true): Seq[v1.RDDStorageInfo]

rddList requests the <> for a[view] over RDDStorageInfoWrapper entities.

In the end, rddList takes RDDStorageInfos with at least one[partition cached] (when cachedOnly flag is on) or all RDDStorageInfos (when cachedOnly flag is off).

NOTE: cachedOnly flag is on and therefore rddList gives RDDs cached only.


rddList is used when:

  • StoragePage is requested to[render] itself

  • AbstractApplicationResource is requested to handle[ storage/rdd] REST API

* StagePagedTable is requested to makeDescription

Last update: 2020-11-29