BlockId

BlockId is the abstraction of block data identifiers that are uniquely identified by a name.

Table 1. BlockId Contract (Abstract Methods Only)
Method Description

name

name: String

Used when…​FIXME

BlockId is a Scala sealed abstract class and so all the possible implementations are in the single Scala file alongside BlockId.
Table 2. BlockIds
BlockId Description

BroadcastBlockId

BlockId for broadcast variables with broadcastId identifier and optional field name (default: empty)

Uses broadcast_ prefix for the name

Used when:

Compressed when spark.broadcast.compress configuration property is enabled

RDDBlockId

BlockId for RDD partitions with rddId and splitIndex identifiers

Uses rdd_ prefix for the name

Used when:

Compressed when spark.rdd.compress configuration property is enabled (default: false)

ShuffleBlockId

BlockId for FIXME with shuffleId, mapId, and reduceId identifiers

Uses shuffle_ prefix for the name

Used when:

Compressed when spark.shuffle.compress configuration property is enabled (default: true)

ShuffleDataBlockId

ShuffleIndexBlockId

StreamBlockId

TaskResultBlockId

TempLocalBlockId

TempShuffleBlockId

Creating BlockId — apply Factory Method

apply(name: String): BlockId

apply simply creates a concrete BlockId by the given name.

apply is used when…​FIXME