DAGSchedulerSource

DAGSchedulerSource is the metrics source of DAGScheduler.

DAGScheduler uses Spark Metrics System to report metrics about internal status.

The name of the source is DAGScheduler.

It emits the following numbers:

  • stage.failedStages - the number of failed stages

  • stage.runningStages - the number of running stages

  • stage.waitingStages - the number of waiting stages

  • job.allJobs - the number of all jobs

  • job.activeJobs - the number of active jobs