Skip to content

AppStatusListener

AppStatusListener is a SparkListener that writes application state information to a data store.

Event Handlers

Event Handler LiveEntities
onJobStart
  • LiveJob
  • LiveStage
  • RDDOperationGraph
onStageSubmitted

Creating Instance

AppStatusListener takes the following to be created:

AppStatusListener is created when:

ElementTrackingStore

AppStatusListener is given an ElementTrackingStore when created.

AppStatusListener registers triggers to clean up state in the store:

ElementTrackingStore is used to write and...FIXME

live Flag

AppStatusListener is given a live flag when created.

live flag indicates whether AppStatusListener is created for the following:

Updating ElementTrackingStore for Active Spark Application

liveUpdate(
  entity: LiveEntity,
  now: Long): Unit

liveUpdate update the ElementTrackingStore when the live flag is enabled.

Updating ElementTrackingStore

update(
  entity: LiveEntity,
  now: Long,
  last: Boolean = false): Unit

update requests the given LiveEntity to write (with the ElementTrackingStore and checkTriggers flag being the given last flag).

getOrCreateExecutor

getOrCreateExecutor(
  executorId: String,
  addTime: Long): LiveExecutor

getOrCreateExecutor...FIXME

getOrCreateExecutor is used when:

getOrCreateStage

getOrCreateStage(
  info: StageInfo): LiveStage

getOrCreateStage...FIXME

getOrCreateStage is used when:


Last update: 2021-06-10