AggregationIterators¶
AggregationIterator
is an abstraction of aggregation iterators of UnsafeRows.
abstract class AggregationIterator(...)
extends Iterator[UnsafeRow]
From scala.collection.Iterator:
Iterators are data structures that allow to iterate over a sequence of elements. They have a
hasNext
method for checking if there is a next element available, and anext
method which returns the next element and discards it from the iterator.
Implementations¶
Creating Instance¶
AggregationIterator
takes the following to be created:
- Partition ID
- Grouping NamedExpressions
- Input Attributes
- AggregateExpressions
- Aggregate Attributes
- Initial input buffer offset
- Result NamedExpressions
- Function to create a new
MutableProjection
given expressions and attributes ((Seq[Expression], Seq[Attribute]) => MutableProjection
)
Abstract Class
AggregationIterator
is an abstract class and cannot be created directly. It is created indirectly for the concrete AggregationIterators.
AggregateModes¶
When created, AggregationIterator
makes sure that there are at most 2 distinct AggregateMode
s of the AggregateExpressions.
The AggregateMode
s have to be a subset of the following mode pairs:
Partial
andPartialMerge
Final
andComplete
AggregateFunctions¶
aggregateFunctions: Array[AggregateFunction]
When created, AggregationIterator
initializes AggregateFunctions in the aggregateExpressions (with initialInputBufferOffset).
initializeAggregateFunctions¶
initializeAggregateFunctions(
expressions: Seq[AggregateExpression],
startingInputBufferOffset: Int): Array[AggregateFunction]
initializeAggregateFunctions
...FIXME
initializeAggregateFunctions
is used when:
AggregationIterator
is requested for the aggregateFunctionsObjectAggregationIterator
is requested for the mergeAggregationBuffersTungstenAggregationIterator
is requested to switchToSortBasedAggregation
generateProcessRow¶
generateProcessRow(
expressions: Seq[AggregateExpression],
functions: Seq[AggregateFunction],
inputAttributes: Seq[Attribute]): (InternalRow, InternalRow) => Unit
generateProcessRow
...FIXME
generateProcessRow
is used when:
AggregationIterator
is requested for the processRow functionObjectAggregationIterator
is requested for the mergeAggregationBuffers functionTungstenAggregationIterator
is requested to switchToSortBasedAggregation
generateOutput¶
generateOutput: (UnsafeRow, InternalRow) => UnsafeRow
When created, AggregationIterator
creates a ResultProjection function.
generateOutput
is used when:
ObjectAggregationIterator
is requested for the next element and to outputForEmptyGroupingKeyWithoutInputSortBasedAggregationIterator
is requested for the next element and to outputForEmptyGroupingKeyWithoutInputTungstenAggregationIterator
is requested for the next element and to outputForEmptyGroupingKeyWithoutInput
generateResultProjection¶
generateResultProjection(): (UnsafeRow, InternalRow) => UnsafeRow
generateResultProjection
...FIXME
initializeBuffer¶
initializeBuffer(
buffer: InternalRow): Unit
initializeBuffer
...FIXME
initializeBuffer
is used when:
SortBasedAggregationIterator
is requested to newBuffer, initialize, next and outputForEmptyGroupingKeyWithoutInput