InterruptibleIterator is a custom Scala Iterator that supports task cancellation, i.e. stops iteration when a task was interrupted (cancelled).
Quoting the official Scala Iterator documentation:
Iterators are data structures that allow to iterate over a sequence of elements. They have a
hasNextmethod for checking if there is a next element available, and a
nextmethod which returns the next element and discards it from the iterator.
InterruptibleIterator is created when:
RDDis requested to get or compute a RDD partition
BlockStoreShuffleReaderis requested to read combined key-value records for a reduce task
PairRDDFunctionsis requested to combineByKeyWithClassTag
JDBCRDDare requested to
RangeExecphysical operator is requested to
BasePythonRunneris requested to
InterruptibleIterator takes the following when created:
In the end,
hasNext requests the delegate Iterator to