Skip to content


RDDCheckpointData is an abstraction of information related to RDD checkpointing.

== [[implementations]] Available RDDCheckpointDatas

[cols="30,70",options="header",width="100%"] |=== | RDDCheckpointData | Description

|[LocalRDDCheckpointData] | [[LocalRDDCheckpointData]]

|[ReliableRDDCheckpointData] | [[ReliableRDDCheckpointData]] Reliable Checkpointing


== [[creating-instance]] Creating Instance

RDDCheckpointData takes the following to be created:

  • [[rdd]][RDD]

== [[Serializable]] RDDCheckpointData as Serializable

RDDCheckpointData is

== [[cpState]] States

  • [[Initialized]] Initialized

  • [[CheckpointingInProgress]] CheckpointingInProgress

  • [[Checkpointed]] Checkpointed

== [[checkpoint]] Checkpointing RDD

[source, scala]

checkpoint(): CheckpointRDD[T]

checkpoint changes the <> to <> only when in <> state. Otherwise, checkpoint does nothing and returns.

checkpoint <> that gives an CheckpointRDD (that is the <> internal registry).

checkpoint changes the <> to <>.

In the end, checkpoint requests the given <> to[markCheckpointed].

checkpoint is used when RDD is requested to[doCheckpoint].

== [[doCheckpoint]] doCheckpoint Method

[source, scala]

doCheckpoint(): CheckpointRDD[T]

doCheckpoint is used when RDDCheckpointData is requested to <>.