JobWaiter[T]( dagScheduler: DAGScheduler, val jobId: Int, totalTasks: Int, resultHandler: (Int, T) => Unit) extends JobListener
You can use a
JobWaiter to block until the job finishes executing or to cancel it.
JobWaiter gets notified about task completions or failures, using
jobFailed, respectively. When the total number of tasks (that equals the number of partitions to compute) equals the number of
JobWaiter instance is marked successful. A
jobFailed event marks the
JobWaiter instance failed.