web UI Configuration Properties¶

spark.ui.enabled¶

Controls whether the web UI is started for the Spark application

Default: true

spark.ui.custom.executor.log.url¶

Specifies custom spark executor log url for supporting external log service instead of using cluster managers' application log urls in the Spark UI. Spark will support some path variables via patterns which can vary on cluster manager. Please check the documentation for your cluster manager to see which patterns are supported, if any. This configuration replaces original log urls in event log, which will be also effective when accessing the application on history server. The new log urls must be permanent, otherwise you might have dead link for executor log urls.

Used when:

• DriverEndpoint is created (and initializes an ExecutorLogUrlHandler)

[[properties]] .web UI Configuration Properties [cols="1,1,2",options="header",width="100%"] |=== | Name | Default Value | Description

Used exclusively when JettyUtils is requested to spark-webui-JettyUtils.md#createServlet[create an HttpServlet].

| [[spark.ui.consoleProgress.update.interval]] spark.ui.consoleProgress.update.interval | 200 (ms) | Update interval, i.e. how often to show the progress.

| [[spark.ui.port]] spark.ui.port | 4040 | The port web UI binds to.

If multiple SparkContexts attempt to run on the same host (it is not possible to have two or more Spark contexts on a single JVM, though), they will bind to successive ports beginning with spark.ui.port.

| [[spark.ui.killEnabled]] spark.ui.killEnabled | true | Enables jobs and stages to be killed from the web UI (true) or not (false).

Used exclusively when SparkUI is requested to spark-webui-SparkUI.md#initialize[initialize] (and registers the redirect handlers for /jobs/job/kill and /stages/stage/kill URIs)

| [[spark.ui.retainedDeadExecutors]] spark.ui.retainedDeadExecutors | 100 | The maximum number of entries in spark-webui-executors-ExecutorsListener.md#executorToTaskSummary[executorToTaskSummary] (in ExecutorsListener) and spark-webui-StorageStatusListener.md#deadExecutorStorageStatus[deadExecutorStorageStatus] (in StorageStatusListener) internal registries.

| [[spark.ui.timeline.executors.maximum]] spark.ui.timeline.executors.maximum | 1000 | The maximum number of entries in <> registry.

| [[spark.ui.timeline.tasks.maximum]] spark.ui.timeline.tasks.maximum | 1000 | |===