Skip to content

PartitionEvaluator¶

PartitionEvaluator[T, U] is an abstraction of partition evaluators that can compute (evaluate) one or more RDD partitions.

Contract¶

Evaluate Partitions¶

eval(
  partitionIndex: Int,
  inputs: Iterator[T]*): Iterator[U]

Used when:

MapPartitionsWithEvaluatorRDD is requested to compute a partition
ZippedPartitionsWithEvaluatorRDD is requested to compute a partition

Implementations¶

Note

No built-in implementations available in Spark Core (but Spark SQL).