SparkListenerEvent¶

SparkListenerEvent is an abstraction of scheduling events.

Dispatching SparkListenerEvents¶

SparkListenerBus in general (and AsyncEventQueue are event buses used to dispatch SparkListenerEvents to registered SparkListeners.

LiveListenerBus is an event bus to dispatch SparkListenerEvents to registered SparkListeners.

Spark History Server¶

Once logged, Spark History Server uses JsonProtocol utility to sparkEventFromJson.

Contract¶

logEvent¶

logEvent: Boolean

logEvent controls whether EventLoggingListener should save the event to an event log.

Default: true

logEvent is used when:

EventLoggingListener is requested to handle "other" events

Implementations¶

SparkListenerApplicationEnd¶

SparkListenerApplicationStart¶

SparkListenerBlockManagerAdded¶

SparkListenerBlockManagerRemoved¶

SparkListenerBlockUpdated¶

SparkListenerEnvironmentUpdate¶

SparkListenerExecutorAdded¶

SparkListenerExecutorBlacklisted¶

SparkListenerExecutorBlacklistedForStage¶

SparkListenerExecutorMetricsUpdate¶

SparkListenerExecutorRemoved¶

SparkListenerExecutorUnblacklisted¶

SparkListenerJobEnd¶

SparkListenerJobStart¶

SparkListenerLogStart¶

SparkListenerNodeBlacklisted¶

SparkListenerNodeBlacklistedForStage¶

SparkListenerNodeUnblacklisted¶

SparkListenerSpeculativeTaskSubmitted¶

SparkListenerStageCompleted¶

SparkListenerStageExecutorMetrics¶

SparkListenerStageSubmitted¶

SparkListenerTaskEnd¶

SparkListenerTaskEnd