Skip to content


LocalSchedulerBackend is a SchedulerBackend and an ExecutorBackend for Spark local deployment.

Master URL Total CPU Cores
local 1
local[n] n
local[*] The number of available CPU cores on the local machine
local[n, m] n CPU cores and m task retries
local[*, m] The number of available CPU cores on the local machine and m task retries

Task status updates flow in local mode

Creating Instance

LocalSchedulerBackend takes the following to be created:

LocalSchedulerBackend is created when:

Maximum Number of Concurrent Tasks

  rp: ResourceProfile): Int

maxNumConcurrentTasks is part of the SchedulerBackend abstraction.

maxNumConcurrentTasks calculates the number of CPU cores per task for the given ResourceProfile (and this SparkConf).

In the end, maxNumConcurrentTasks is the total CPU cores available divided by the number of CPU cores per task.


Enable ALL logging level for org.apache.spark.scheduler.local.LocalSchedulerBackend logger to see what happens inside.

Add the following line to conf/ = org.apache.spark.scheduler.local.LocalSchedulerBackend
logger.LocalSchedulerBackend.level = all

Refer to Logging.