StageInfo¶

StageInfo is a metadata about a stage to pass from the scheduler to SparkListeners.

Creating Instance¶

StageInfo takes the following to be created:

• Stage ID
• Stage Attempt ID
• Name
• Number of Tasks
• RDDInfos
• Parent IDs
• Details
• TaskMetrics (default: null)
• Task Locality Preferences (default: empty)
• Optional Shuffle Dependency ID (default: undefined)

StageInfo is created when:

• StageInfo utility is used to fromStage
• JsonProtocol (History Server) is used to stageInfoFromJson

fromStage Utility¶

fromStage(
  stage: Stage,
  attemptId: Int,
  numTasks: Option[Int] = None,
  taskMetrics: TaskMetrics = null,
  taskLocalityPreferences: Seq[Seq[TaskLocation]] = Seq.empty): StageInfo

fromStage...FIXME

fromStage is used when:

• Stage is created and make a new Stage attempt