Skip to content

AppStatusListener

AppStatusListener is a SparkListener.

Creating Instance

AppStatusListener takes the following to be created:

AppStatusListener is created when:

live Flag

AppStatusListener is given a live flag when created.

live flag indicates whether AppStatusListener is created for the following:

onJobStart

onJobStart(
  event: SparkListenerJobStart): Unit

onJobStart...FIXME

onJobStart is part of the SparkListener abstraction.

onStageSubmitted

onStageSubmitted(
  event: SparkListenerStageSubmitted): Unit

onStageSubmitted is part of the SparkListener abstraction.

onStageSubmitted getOrCreateStage for the StageInfo (from the given SparkListenerStageSubmitted event).

onStageSubmitted...FIXME

In the end, onStageSubmitted liveUpdate with the stage and now timestamp.

Updating ElementTrackingStore for Active Spark Application

liveUpdate(
  entity: LiveEntity,
  now: Long): Unit

liveUpdate update the ElementTrackingStore when the live flag is enabled.

Updating ElementTrackingStore

update(
  entity: LiveEntity,
  now: Long,
  last: Boolean = false): Unit

update requests the given LiveEntity to write (with the ElementTrackingStore and checkTriggers flag being the given last flag).

getOrCreateStage

getOrCreateStage(
  info: StageInfo): LiveStage

getOrCreateStage...FIXME

getOrCreateStage is used when AppStatusListener is requested to onJobStart and onStageSubmitted.


Last update: 2020-11-29