ExecutorMetricsSource¶

ExecutorMetricsSource is a metrics source.

Creating Instance¶

ExecutorMetricsSource takes no arguments to be created.

ExecutorMetricsSource is created when:

• SparkContext is created (with spark.metrics.executorMetricsSource.enabled enabled)
• Executor is created (with spark.metrics.executorMetricsSource.enabled enabled)

Source Name¶

sourceName: String

sourceName is ExecutorMetrics.

sourceName is part of the Source abstraction.

Registering with MetricsSystem¶

register(
  metricsSystem: MetricsSystem): Unit

register creates ExecutorMetricGauges for every executor metric.

register requests the MetricRegistry to register every metric type.

In the end, register requests the MetricRegistry to register this ExecutorMetricsSource.

register is used when:

• SparkContext is created
• Executor is created (for non-local mode)

Metrics Snapshot¶

ExecutorMetricsSource defines metricsSnapshot internal registry of values of every metric.

The values are updated in updateMetricsSnapshot and read using ExecutorMetricGauges.

updateMetricsSnapshot¶

updateMetricsSnapshot(
  metricsUpdates: Array[Long]): Unit

updateMetricsSnapshot updates the metricsSnapshot registry with the given metricsUpdates.

updateMetricsSnapshot is used when:

• SparkContext is requested to reportHeartBeat
• ExecutorMetricsPoller is requested to poll