Skip to content

== Executors Tab

Executors tab in[web UI] shows...FIXME

.Executors Tab in web UI (local mode) image::spark-webui-executors.png[align="center"]

The Executors tab is available under /executors URL, i.e. http://localhost:4040/executors.

Internally, the Executors tab is represented by[ExecutorsTab].


What's interesting in how Storage Memory is displayed in the Executors tab is that the default UnifiedMemoryManager[calculates the maximum memory] in a way that is different from what the page displays (using the custom JavaScript formatBytes function in utils.js).

[source, scala]

// local mode with spark.driver.memory 2g // ./bin/spark-shell --conf spark.driver.memory=2g // UnifiedMemoryManager reports 912MB // You can see it after enabling INFO messages for BlockManagerMasterEndpoint

INFO BlockManagerMasterEndpoint: Registering block manager with 912.3 MB RAM, BlockManagerId(driver,, 54503, None)

// custom JavaScript formatBytes function (from utils.js) reports...956.6MB // See def formatBytes(bytes: Double) = { val k = 1000 val i = math.floor(math.log(bytes) / math.log(k)) val maxMemoryWebUI = bytes / math.pow(k, i) f"$maxMemoryWebUI%1.1f" } scala> println(formatBytes(maxMemory)) 956.6


=== [[getExecInfo]] getExecInfo Method

[source, scala]

getExecInfo( listener: ExecutorsListener, statusId: Int, isActive: Boolean): ExecutorSummary

getExecInfo creates a ExecutorSummary.


NOTE: getExecInfo is used when...FIXME

=== [[settings]] Settings

==== [[spark_ui_threadDumpsEnabled]] spark.ui.threadDumpsEnabled

spark.ui.threadDumpsEnabled (default: true) is to enable (true) or disable (false) <>.

Last update: 2020-10-06