Skip to content

ExecutorAllocationClient

ExecutorAllocationClient is an abstraction of schedulers that can communicate with a cluster manager to request or kill executors.

Implementations

Active Executors

getExecutorIds(): Seq[String]

Used when:

Whether Executor is Active

isExecutorActive(
  id: String): Boolean

Returns whether a given executor is active (and can be used to execute tasks)

Killing Executors

killExecutors(
  executorIds: Seq[String],
  adjustTargetNumExecutors: Boolean,
  countFailures: Boolean,
  force: Boolean = false): Seq[String]

Requests a cluster manager to kill given executors and returns whether the request has been acknowledged by the cluster manager (true) or not (false).

Used when:

Killing Executors on Host

killExecutorsOnHost(
  host: String): Boolean

Used when:

Requesting Additional Executors

requestExecutors(
  numAdditionalExecutors: Int): Boolean

Requests additional executors from a cluster manager and returns whether the request has been acknowledged by the cluster manager (true) or not (false).

Used when:

Updating Total Executors

requestTotalExecutors(
  numExecutors: Int,
  localityAwareTasks: Int,
  hostToLocalTaskCount: Map[String, Int]): Boolean

Updates a cluster manager with the exact number of executors desired. Returns whether the request has been acknowledged by the cluster manager (true) or not (false).

Used when:

Killing Executor

killExecutor(
  executorId: String): Boolean

killExecutor kill the given executor.

killExecutor is used when:


Last update: 2021-01-05