LocalEndpoint is the communication channel between TaskSchedulerImpl and LocalSchedulerBackend.
LocalEndpoint is a (thread-safe) RpcEndpoint that hosts an executor (with
driver ID and
localhost hostname) for Spark local mode.
LocalEndpoint starts up (as part of Spark local’s initialization) it prints out the following INFO messages to the logs:
INFO Executor: Starting executor ID driver on host localhost INFO Executor: Using REPL class URI: http://192.168.1.4:56131
LocalEndpoint creates a single Executor with the following properties:
Default: Initial number of CPU cores (aka totalCores)
Add the following line to
Refer to Logging.
LocalEndpoint takes the following to be created:
LocalEndpoint initializes the internal registries and counters.
receive: PartialFunction[Any, Unit]
StatusUpdate( taskId: Long, state: TaskState, serializedData: ByteBuffer)
If the given TaskState is a finished state (one of
LocalEndpoint adds spark.task.cpus configuration (default:
1) to the freeCores registry followed by reviveOffers.
KillTask( taskId: Long, interruptThread: Boolean, reason: String)
receiveAndReply(context: RpcCallContext): PartialFunction[Any, Unit]
receiveAndReply handles (processes) StopExecutor RPC message exclusively.