Skip to content

BroadcastManager

BroadcastManager manages a TorrentBroadcastFactory.

BroadcastManager, SparkEnv and BroadcastFactory

Note

As of Spark 2.0, it is no longer possible to plug a custom BroadcastFactory in, and TorrentBroadcastFactory is the only known implementation.

Creating Instance

BroadcastManager takes the following to be created:

While being created, BroadcastManager is requested to initialize.

BroadcastManager is created when:

Initializing

initialize(): Unit

Unless initialized already, initialize creates a TorrentBroadcastFactory and requests it to initialize itself.

TorrentBroadcastFactory

BroadcastManager manages a BroadcastFactory:

  • Creates and initializes it when created (and requested to initialize)

  • Stops it when stopped

BroadcastManager uses the BroadcastFactory when requested for the following:

Creating Broadcast Variable

newBroadcast(
  value_ : T,
  isLocal: Boolean): Broadcast[T]

newBroadcast requests the BroadcastFactory for a new broadcast variable (with the next available broadcast ID).

newBroadcast is used when:

Unique Identifiers of Broadcast Variables

BroadcastManager tracks broadcast variables and assigns unique and continuous identifiers.

MapOutputTrackerMaster

BroadcastManager is used to create a MapOutputTrackerMaster

Deleting Broadcast Variable

unbroadcast(
  id: Long,
  removeFromDriver: Boolean,
  blocking: Boolean): Unit

unbroadcast requests the BroadcastFactory to delete a broadcast variable (by id).

unbroadcast is used when: