SparkFiles¶

SparkFiles is an utility to work with files added using SparkContext.addFile.

Absolute Path of Added File¶

get(
  filename: String): String

get gets the absolute path of the given file in the root directory.

Root Directory¶

getRootDirectory(): String

getRootDirectory requests the current SparkEnv for driverTmpDir (if defined) or defaults to the current directory (.).

getRootDirectory is used when:

• SparkContext is requested to addFile
• Executor is requested to updateDependencies
• SparkFiles utility is requested to get the absolute path of a file