RDDBarrier¶

RDDBarrier is a wrapper around RDD with two custom map transformations:

• mapPartitions
• mapPartitionsWithIndex

Unlike regular RDD.mapPartitions transformations, RDDBarrier transformations create a MapPartitionsRDD with isFromBarrier flag enabled.

RDDBarrier (of T records) marks the current stage as a barrier stage in Barrier Execution Mode.

Creating Instance¶

RDDBarrier takes the following to be created:

• RDD (of T records)

RDDBarrier is created when:

• RDD.barrier transformation is used