SparkSubmitOperation¶

SparkSubmitOperation is an abstraction of operations of spark-submit (when requested to kill a submission or for a submission status).

Contract¶

Killing Submission¶

kill(
  submissionId: String,
  conf: SparkConf): Unit

Kills a given submission

Used when:

• SparkSubmit is requested to kill a submission

Displaying Submission Status¶

printSubmissionStatus(
  submissionId: String,
  conf: SparkConf): Unit

Displays status of a given submission

Used when:

• SparkSubmit is requested for submission status

Checking Whether Master URL Supported¶

supports(
  master: String): Boolean

Used when:

• SparkSubmit is requested to kill a submission and for a submission status (via getSubmitOperations utility)

Implementations¶

• K8SSparkSubmitOperation (Spark on Kubernetes)