BufferedRowIterator¶

BufferedRowIterator is an abstraction of iterators that are code-generated at execution time in Whole-Stage Java Code Generation (using WholeStageCodegenExec unary physical operator).

Contract¶

Initializing¶

void init(
  int index,
  Iterator<InternalRow>[] iters)

Used when WholeStageCodegenExec unary physical operator is executed

Processing Next Row¶

void processNext()

Used when BufferedRowIterator is requested to hasNext

hasNext¶

boolean hasNext()

hasNext processNext when there are no rows in the currentRows buffer.

hasNext is true when the currentRows buffer has got any element. Otherwise, hasNext is false.

hasNext is used when WholeStageCodegenExec unary physical operator is executed (to generate an RDD[InternalRow] to mapPartitionsWithIndex over rows from up to two input RDDs).

currentRows Buffer¶

LinkedList<InternalRow> currentRows

currentRows is an internal buffer of InternalRow:

• A new row is added when appending a row
• A row is removed when requested to next

currentRows is used for hasNext and shouldStop.

Appending Row¶

void append(
  InternalRow row)

append simply adds the given InternalRow to the end of the currentRows buffer (thus the name append).

append is used...FIXME