ShuffleReader¶
ShuffleReader
is a <
[[contract]] [source, scala]
package org.apache.spark.shuffle
trait ShuffleReader[K, C] { def read(): Iterator[Product2[K, C]] }
NOTE: ShuffleReader
is a private[spark]
contract.
.ShuffleReader Contract [cols="1,2",options="header",width="100%"] |=== | Method | Description
| read
a| [[read]] Reading combined key-value records for a reduce task
Used when:
-
CoGroupedRDD, ShuffledRDD, and SubtractedRDD are requested to compute a partition (for a
ShuffleDependency
dependency) -
Spark SQL's
ShuffledRowRDD
is requested tocompute
a partition |===
[[implementations]] NOTE: shuffle:BlockStoreShuffleReader.md[BlockStoreShuffleReader] is the one and only known <
Last update: 2020-10-09