BroadcastFactory

BroadcastFactory is an abstraction for factories that BroadcastManager uses for Broadcast Variable.

As of Spark 2.0, it is no longer possible to plug a custom BroadcastFactory in, and TorrentBroadcastFactory is the one and only known implementation.

Contract

initialize Method

initialize(
  isDriver: Boolean,
  conf: SparkConf,
  securityMgr: SecurityManager): Unit

Used when BroadcastManager is created.

newBroadcast Method

newBroadcast[T: ClassTag](
  value: T,
  isLocal: Boolean,
  id: Long): Broadcast[T]

Used when BroadcastManager is requested for a new broadcast variable.

stop Method

stop(): Unit

Used when BroadcastManager is requested to stop.

unbroadcast Method

unbroadcast(
  id: Long,
  removeFromDriver: Boolean,
  blocking: Boolean): Unit

Used when BroadcastManager is requested to unbroadcast a broadcast variable.

Available BroadcastFactories

TorrentBroadcastFactory is the default and only known BroadcastFactory in Apache Spark.