Skip to content

web UI Configuration Properties

spark.ui.custom.executor.log.url

Specifies custom spark executor log url for supporting external log service instead of using cluster managers' application log urls in the Spark UI. Spark will support some path variables via patterns which can vary on cluster manager. Please check the documentation for your cluster manager to see which patterns are supported, if any. This configuration replaces original log urls in event log, which will be also effective when accessing the application on history server. The new log urls must be permanent, otherwise you might have dead link for executor log urls.

Used when:

spark.ui.enabled

Controls whether the web UI is started for the Spark application

Default: true

spark.ui.port

The port the web UI of a Spark application binds to

Default: 4040

If multiple SparkContexts attempt to run on the same host (as different Spark applications), they will bind to successive ports beginning with spark.ui.port (until spark.port.maxRetries).

Used when:

spark.ui.prometheus.enabled

internal Expose executor metrics at /metrics/executors/prometheus

Default: false

Used when:

Review Me

[[properties]] .web UI Configuration Properties [cols="1,1,2",options="header",width="100%"] |=== | Name | Default Value | Description

[[spark.ui.allowFramingFrom]] spark.ui.allowFramingFrom
Defines the URL to use in ALLOW-FROM in X-Frame-Options header (as described in http://tools.ietf.org/html/rfc7034).

Used exclusively when JettyUtils is requested to spark-webui-JettyUtils.md#createServlet[create an HttpServlet].

| [[spark.ui.consoleProgress.update.interval]] spark.ui.consoleProgress.update.interval | 200 (ms) | Update interval, i.e. how often to show the progress.

| [[spark.ui.killEnabled]] spark.ui.killEnabled | true | Enables jobs and stages to be killed from the web UI (true) or not (false).

Used exclusively when SparkUI is requested to spark-webui-SparkUI.md#initialize[initialize] (and registers the redirect handlers for /jobs/job/kill and /stages/stage/kill URIs)

| [[spark.ui.retainedDeadExecutors]] spark.ui.retainedDeadExecutors | 100 |

| [[spark.ui.timeline.executors.maximum]] spark.ui.timeline.executors.maximum | 1000 | The maximum number of entries in <> registry.

| [[spark.ui.timeline.tasks.maximum]] spark.ui.timeline.tasks.maximum | 1000 | |===


Last update: 2021-06-09
Back to top