Skip to content

SparkListenerEvent

SparkListenerEvent is an abstraction of scheduling events.

Dispatching SparkListenerEvents

SparkListenerBus in general (and AsyncEventQueue in particular) are event buses used to dispatch SparkListenerEvents to registered SparkListeners.

LiveListenerBus is an event bus to dispatch SparkListenerEvents to registered SparkListeners.

Spark History Server

Once logged, Spark History Server uses JsonProtocol utility to sparkEventFromJson.

Contract

logEvent

logEvent: Boolean

logEvent controls whether EventLoggingListener should save the event to an event log.

Default: true

logEvent is used when:

Implementations

SparkListenerApplicationEnd

SparkListenerApplicationStart

SparkListenerBlockManagerAdded

SparkListenerBlockManagerRemoved

SparkListenerBlockUpdated

SparkListenerEnvironmentUpdate

SparkListenerExecutorAdded

SparkListenerExecutorBlacklisted

SparkListenerExecutorBlacklistedForStage

SparkListenerExecutorMetricsUpdate

SparkListenerExecutorRemoved

SparkListenerExecutorUnblacklisted

SparkListenerJobEnd

SparkListenerJobStart

SparkListenerLogStart

SparkListenerNodeBlacklisted

SparkListenerNodeBlacklistedForStage

SparkListenerNodeUnblacklisted

SparkListenerSpeculativeTaskSubmitted

SparkListenerStageCompleted

SparkListenerStageExecutorMetrics

SparkListenerStageSubmitted

SparkListenerTaskEnd

SparkListenerTaskGettingResult

SparkListenerTaskStart

SparkListenerUnpersistRDD


Last update: 2020-12-31