BlockDataManager

BlockDataManager is the abstraction of block data managers that manage storage for blocks of data (aka block storage management API).

BlockDataManager uses BlockIds to uniquely identify blocks of data and ManagedBuffer to refer to them.

BlockManager is the default and only implementation of the BlockDataManager Contract in Apache Spark.
Table 1. BlockDataManager Contract
Method Description

getBlockData

getBlockData(blockId: BlockId): ManagedBuffer

Fetches a block data (as a ManagedBuffer) for the given BlockId

Used when:

putBlockData

putBlockData(
  blockId: BlockId,
  data: ManagedBuffer,
  level: StorageLevel,
  classTag: ClassTag[_]): Boolean

Stores (puts) a block data (as a ManagedBuffer) for the given BlockId. Returns true when completed successfully or false when failed.

Used when NettyBlockRpcServer is requested to handle an UploadBlock message

putBlockDataAsStream

putBlockDataAsStream(
  blockId: BlockId,
  level: StorageLevel,
  classTag: ClassTag[_]): StreamCallbackWithID

Stores a block data that will be received as a stream

Used when NettyBlockRpcServer is requested to receiveStream

releaseLock

releaseLock(
  blockId: BlockId,
  taskAttemptId: Option[Long]): Unit

Releases a lock

Used when: