== [[AbstractCommandBuilder]] AbstractCommandBuilder
AbstractCommandBuilder is the base command builder for spark-submit-SparkSubmitCommandBuilder.md[SparkSubmitCommandBuilder] and
SparkClassCommandBuilder specialized command builders.
AbstractCommandBuilder expects that command builders define
AbstractCommandBuilder Methods [cols="1,2",options="header",width="100%"] |=== | Method | Description |
buildCommand | The only abstract method that subclasses have to define. | <
spark-defaults.conf file under the Spark configuration directory. |===
buildJavaCommand Internal Method
buildJavaCommand builds the Java command for a Spark application (which is a collection of elements with the path to
java executable, JVM options from
java-opts file, and a class path).
javaHome is set,
[javaHome]/bin/java to the result Java command. Otherwise, it uses
JAVA_HOME or, when no earlier checks succeeded, falls through to
java.home Java's system property.
CAUTION: FIXME Who sets
javaHome internal property and when?
buildJavaCommand loads extra Java options from the
java-opts file in <
-cp to the result Java command.
buildClassPath builds the classpath for a Spark application.
NOTE: Directories always end up with the OS-specific file separator at the end of their paths.
buildClassPath adds the following in that order:
- The input
- The spark-AbstractCommandBuilder.md#getConfDir[configuration directory]
1) Locally compiled Spark classes in
test-classesand Core's jars. + CAUTION: FIXME Elaborate on "locally compiled Spark classes".
1) ... + CAUTION: FIXME Elaborate on the SQL testing case
childEnv is queried first before System properties. It is always empty for
=== [[loadPropertiesFile]] Loading Properties File --
loadPropertiesFile Internal Method
loadPropertiesFile is part of
AbstractCommandBuilder private API that loads Spark settings from a properties file (when specified on the command line) or spark-properties.md#spark-defaults-conf[spark-defaults.conf] in the <
It loads the settings from the following files starting from the first and checking every location until the first properties file is found:
propertiesFile(if specified using
--properties-filecommand-line option or set by
loadPropertiesFile reads a properties file using
=== [[getConfDir]][[configuration-directory]] Spark's Configuration Directory --
getConfDir Internal Method
getConfDir to compute the current configuration directory of a Spark application.
childEnv which is always empty anyway or as a environment variable) and falls through to
SPARK_HOME from <
=== [[getSparkHome]][[home-directory]] Spark's Home Directory --
getSparkHome Internal Method
getSparkHome to compute Spark's home directory for a Spark application.
childEnv which is always empty anyway or as a environment variable).
SPARK_HOME is not set, Spark throws a
Spark home not found; set it explicitly or use the SPARK_HOME environment variable.