YarnClusterManager

YarnClusterManager is the only currently known ExternalClusterManager in Spark. It creates a TaskScheduler and a SchedulerBackend for YARN.

canCreate Method

YarnClusterManager can handle the yarn master URL only.

createTaskScheduler Method

It throws a SparkException for unknown deploy modes.

Unknown deploy mode '[deployMode]' for Yarn

createSchedulerBackend Method

It throws a SparkException for unknown deploy modes.

Unknown deploy mode '[deployMode]' for Yarn

Initializing YarnClusterManager — initialize Method