ExecutorBackend is a pluggable interface that TaskRunners use to send task status updates to a scheduler.

Figure 1. ExecutorBackend receives notifications from TaskRunners
TaskRunner manages a single individual task and is managed by an Executor to launch a task.

It is effectively a bridge between the driver and an executor, i.e. there are two endpoints running.

There are three concrete executor backends:

ExecutorBackend Contract

statusUpdate Method

  taskId: Long,
  state: TaskState,
  data: ByteBuffer): Unit

Used when TaskRunner is requested to run a task (to send task status updates).