MigratableResolver¶

MigratableResolver is an abstraction of resolvers that allow Spark to migrate shuffle blocks.

Contract¶

getMigrationBlocks¶

getMigrationBlocks(
  shuffleBlockInfo: ShuffleBlockInfo): List[(BlockId, ManagedBuffer)]

Used when:

• ShuffleMigrationRunnable is requested to run

getStoredShuffles¶

getStoredShuffles(): Seq[ShuffleBlockInfo]

Used when:

• BlockManagerDecommissioner is requested to refreshOffloadingShuffleBlocks

putShuffleBlockAsStream¶

putShuffleBlockAsStream(
  blockId: BlockId,
  serializerManager: SerializerManager): StreamCallbackWithID

Used when:

• BlockManager is requested to putBlockDataAsStream

Implementations¶

• IndexShuffleBlockResolver