RDDCheckpointData

RDDCheckpointData is an abstraction of information related to RDD checkpointing.

Available RDDCheckpointDatas

Creating Instance

RDDCheckpointData takes the following to be created:

RDDCheckpointData as Serializable

RDDCheckpointData is java.io.Serializable.

States

  • Initialized

  • CheckpointingInProgress

  • Checkpointed

Checkpointing RDD

checkpoint(): CheckpointRDD[T]

checkpoint changes the state to CheckpointingInProgress only when in Initialized state. Otherwise, checkpoint does nothing and returns.

checkpoint doCheckpoint that gives an CheckpointRDD (that is the cpRDD internal registry).

checkpoint changes the state to Checkpointed.

In the end, checkpoint requests the given RDD to markCheckpointed.

checkpoint is used when RDD is requested to doCheckpoint.

doCheckpoint Method

doCheckpoint(): CheckpointRDD[T]

doCheckpoint is used when RDDCheckpointData is requested to checkpoint the RDD.