ClientDistributedCacheManager is a mere wrapper to hold the collection of cache-related resource entries
distCacheEntries) to add resources to and later update Spark configuration with files to distribute.
|FIXME What is a resource? Is this a file only?|
addResource( fs: FileSystem, conf: Configuration, destPath: Path, localResources: HashMap[String, LocalResource], resourceType: LocalResourceType, link: String, statCache: Map[URI, FileStatus], appMasterOnly: Boolean = false): Unit
updateConfiguration(conf: SparkConf): Unit
updateConfiguration sets the following internal Spark configuration settings in the input
conf Spark configuration:
It uses the internal
distCacheEntries with resources to distribute.
It is later used in