YarnRMClient tracks the application attempt identifiers and the maximum number of attempts to register
Created (initialized and started) when
Add the following line to
Refer to Logging.
Registering ApplicationMaster with YARN ResourceManager (and Creating YarnAllocator) —
register( driverUrl: String, driverRef: RpcEndpointRef, conf: YarnConfiguration, sparkConf: SparkConf, uiAddress: String, uiHistoryAddress: String, securityMgr: SecurityManager, localResources: Map[String, LocalResource]): YarnAllocator
register saves the input
uiHistoryAddress as uiHistoryAddress.
You should see the following INFO message in the logs (in stderr in YARN):
INFO YarnRMClient: Registering the ApplicationMaster
register then uses AMRMClient to register the Spark application’s
ApplicationMaster (using the local hostname, the port
0 and the input
registered flag is enabled.
unregister(status: FinalApplicationStatus, diagnostics: String = ""): Unit
unregister unregisters the ApplicationMaster of a Spark application.
unregister is called when
ApplicationMaster wants to unregister.
getMaxRegAttempts(sparkConf: SparkConf, yarnConf: YarnConfiguration): Int
getMaxRegAttempts uses SparkConf and YARN’s YarnConfiguration to read configuration settings and return the maximum number of application attempts before ApplicationMaster registration with YARN is considered unsuccessful (and so the Spark application).
The return value is the minimum of the configuration settings of YARN and Spark.
getAttemptId returns YARN’s
ApplicationAttemptId (of the Spark application to which the container was assigned).