ShuffleDataIO¶

ShuffleDataIO is an abstraction of pluggable temporary shuffle block store plugins for storing shuffle blocks in arbitrary storage backends.

Contract¶

ShuffleDriverComponents¶

ShuffleDriverComponents driver()

Used when:

• SparkContext is created

ShuffleExecutorComponents¶

ShuffleExecutorComponents executor()

Used when:

• SortShuffleManager utility is used to load the ShuffleExecutorComponents

Implementations¶

• LocalDiskShuffleDataIO