AppStatusStore

AppStatusStore is…​FIXME

AppStatusStore createLiveStore
Figure 1. AppStatusStore in Spark Application

AppStatusStore is available as SparkContext.statusStore to other Spark services.

Creating Instance

AppStatusStore takes the following to be created:

AppStatusStore is created when:

streamBlocksList Method

streamBlocksList(): Seq[StreamBlockData]

streamBlocksList…​FIXME

streamBlocksList is used when…​FIXME

activeStages Method

activeStages(): Seq[v1.StageData]

activeStages…​FIXME

activeStages is used when…​FIXME

Creating Event Store

createLiveStore(
  conf: SparkConf): AppStatusStore

createLiveStore creates a fully-initialized AppStatusStore.

Internally, createLiveStore creates a ElementTrackingStore (with a new InMemoryStore and the input SparkConf).

createLiveStore creates a AppStatusListener (with the ElementTrackingStore created, the input SparkConf and the live flag enabled).

In the end, createLiveStore creates an AppStatusStore (with the ElementTrackingStore and AppStatusListener just created).

createLiveStore is used when SparkContext is created.

Closing AppStatusStore

close(): Unit

close simply requests KVStore to close.

close is used when…​FIXME

rddList Method

rddList(cachedOnly: Boolean = true): Seq[v1.RDDStorageInfo]

rddList requests the KVStore for a view over RDDStorageInfoWrapper entities.

In the end, rddList takes RDDStorageInfos with at least one partition cached (when cachedOnly flag is on) or all RDDStorageInfos (when cachedOnly flag is off).

cachedOnly flag is on and therefore rddList gives RDDs cached only.

rddList is used when:

  • StoragePage is requested to render itself

  • AbstractApplicationResource is requested to handle storage/rdd REST API

  • StagePagedTable is requested to makeDescription