Skip to content

ShuffleReader

ShuffleReader is a <> of <> to <>.

[[contract]] [source, scala]


package org.apache.spark.shuffle

trait ShuffleReader[K, C] { def read(): Iterator[Product2[K, C]] }


NOTE: ShuffleReader is a private[spark] contract.

.ShuffleReader Contract [cols="1,2",options="header",width="100%"] |=== | Method | Description

| read a| [[read]] Reading combined key-value records for a reduce task

Used when:

  • CoGroupedRDD, ShuffledRDD, and SubtractedRDD are requested to compute a partition (for a ShuffleDependency dependency)

  • Spark SQL's ShuffledRowRDD is requested to compute a partition |===

[[implementations]] NOTE: shuffle:BlockStoreShuffleReader.md[BlockStoreShuffleReader] is the one and only known <> in Apache Spark.


Last update: 2020-10-09