Broadcast Manager (BroadcastManager) is a Spark service to manage broadcast variables in Spark. It is created for a Spark application when SparkContext is initialized and is a simple wrapper around BroadcastFactory.

BroadcastManager tracks the number of broadcast variables in a Spark application (using the internal field nextBroadcastId).

The idea is to transfer values used in transformations from a driver to executors in a most effective way so they are copied once and used many times by tasks (rather than being copied every time a task is launched).

When initialized, BroadcastManager creates an instance of TorrentBroadcastFactory.

stop Method


Creating BroadcastManager Instance


Initializing BroadcastManager — initialize Internal Method

initialize(): Unit
initialize is executed only once (when BroadcastManager is created) and controlled by the internal initialized flag.

newBroadcast Method

newBroadcast[T](value_ : T, isLocal: Boolean): Broadcast[T]

newBroadcast simply requests the current BroadcastFactory for a new broadcast variable.

The BroadcastFactory is created when BroadcastManager is initialized.


Table 1. Settings
Name Default value Description



The size of a block (in kB when unit not specified).

Used when TorrentBroadcast stores brodcast blocks to BlockManager.



The flag to enable compression.

Refer to CompressionCodec.

Used when TorrentBroadcast is created and later when it stores broadcast blocks to BlockManager. Also in SerializerManager.