DeclarativeAggregate Expression-Based Functions¶

DeclarativeAggregate is an extension of the AggregateFunction abstraction for expression-based aggregate functions that use Catalyst Expression for evaluation.

DeclarativeAggregate is an Unevaluable.

Contract¶

Evaluating Expression¶

evaluateExpression: Expression

Catalyst Expression to calculate the final value of this aggregate function

See:

• RowNumber

Used when:

• EliminateAggregateFilter logical optimization is executed
• AggregatingAccumulator is created
• AggregationIterator is requested for the generateResultProjection
• HashAggregateExec physical operator is requested to doProduceWithoutKeys and generateResultFunction
• AggregateProcessor is created

Expressions to Initialize Empty Aggregation Buffers¶

initialValues: Seq[Expression]

Catalyst Expressions to initialize empty aggregation buffers (for the initial values of this aggregate function)

See:

• Count
• SimpleTypedAggregateExpression

Used when:

• EliminateAggregateFilter logical optimization is executed
• AggregatingAccumulator is created
• AggregateCodegenSupport is requested to doProduceWithoutKeys
• AggregationIterator is created
• HashAggregateExec physical operator is requested to createHashMap, getEmptyAggregationBuffer
• HashMapGenerator is created
• AggregateProcessor is created

Merge Expressions¶

mergeExpressions: Seq[Expression]

Catalyst Expressions

See:

• Count
• SimpleTypedAggregateExpression

Update Expressions¶

updateExpressions: Seq[Expression]

Catalyst Expressions to update the mutable aggregation buffer based on an input row

See:

• Count
• RowNumber
• SimpleTypedAggregateExpression

Used when:

• AggregateProcessor is created
• AggregateCodegenSupport is requested to doConsumeWithoutKeys
• AggregationIterator is requested to generateProcessRow
• AggregatingAccumulator is created
• HashAggregateExec is requested to doConsumeWithKeys

Implementations¶

• AggregateWindowFunction
• First
• others