Skip to content

Configuration Properties of Kubernetes Cluster Manager

spark.kubernetes.submitInDriver

(internal)

Default: false

spark.kubernetes.driver.pod.name

Name of the driver pod

Default: (undefined)

Must be provided if a Spark application is deployed using spark-submit in cluster mode

spark.kubernetes.executor.podNamePrefix

(internal) Prefix to use in front of the executor pod names

Default: (undefined)

spark.kubernetes.namespace

The namespace that will be used for running the driver and executor pods.

Default: default

spark.kubernetes.executor.podTemplateFile

File containing a template pod spec for executors

Default: (undefined)

spark.kubernetes.executor.podTemplateContainerName

Container name to be used as a basis for executors in the given pod template

Default: (undefined)

spark.kubernetes.authenticate.driver.mounted

FIXME

spark.kubernetes.driver.master

The internal Kubernetes master (API server) address to be used for driver to request executors.

Default: https://kubernetes.default.svc

spark.kubernetes.authenticate

FIXME

spark.kubernetes.executor.eventProcessingInterval

Interval between successive inspection of executor events sent from the Kubernetes API

Default: 1s

spark.kubernetes.executor.deleteOnTermination

When disabled (false), executor pods will not be deleted in case of failure or normal termination

Default: true


Last update: 2020-10-10