ColumnarToRowExec Physical Operator¶
ColumnarToRowExec
is a unary physical operator for Columnar Processing.
ColumnarToRowExec
supports Whole-Stage Java Code Generation.
Creating Instance¶
ColumnarToRowExec
takes the following to be created:
- Child physical operator
ColumnarToRowExec
requires that the child physical operator supportsColumnar.
ColumnarToRowExec
is created when ApplyColumnarRulesAndInsertTransitions physical optimization is executed.
Performance Metrics¶
Key | Name (in web UI) | Description |
---|---|---|
numInputBatches | number of input batches | Number of input batches |
numOutputRows | number of output rows | Number of output rows (across all input batches) |
Executing Physical Operator¶
doExecute(): RDD[InternalRow]
doExecute
is part of the SparkPlan abstraction.
doExecute
requests the child physical operator to executeColumnar and RDD.mapPartitionsInternal
over batches (Iterator[ColumnarBatch]
) to "unpack" to rows. doExecute
counts the number of batches and rows (as the metrics).
Input RDDs¶
inputRDDs(): Seq[RDD[InternalRow]]
inputRDDs
is a single RDD[ColumnarBatch]
that the child physical operator gives when requested to executeColumnar.
inputRDDs
is part of the CodegenSupport abstraction.
canCheckLimitNotReached Flag¶
canCheckLimitNotReached: Boolean
canCheckLimitNotReached
is always true
.
canCheckLimitNotReached
is part of the CodegenSupport abstraction.