Skip to content


MapPartitionsRDD[U, T] is a RDD that transforms (maps) input T records into Us using partition function.

MapPartitionsRDD is a RDD that has exactly one-to-one narrow dependency on the parent RDD.

Creating Instance

MapPartitionsRDD takes the following to be created:

  • Parent RDD (RDD[T])
  • Partition Function
  • preservesPartitioning flag
  • isFromBarrier Flag
  • isOrderSensitive flag

MapPartitionsRDD is created when:

Barrier RDD

MapPartitionsRDD can be a barrier RDD in Barrier Execution Mode.

isFromBarrier Flag

MapPartitionsRDD can be given isFromBarrier flag when created.

isFromBarrier flag is assumed disabled (false) and can only be enabled (true) using RDDBarrier transformations:


isBarrier_ : Boolean

isBarrier_ is part of the RDD abstraction.

isBarrier_ is enabled (true) when either this MapPartitionsRDD is isFromBarrier or any of the parent RDDs is isBarrier. Otherwise, isBarrier_ is disabled (false).