Skip to content

AggregateFunction Expressions

AggregateFunction is an extension of the Expression abstraction for aggregate functions.

AggregateFunction can never be foldable.


Aggregation Buffer Attributes

aggBufferAttributes: Seq[AttributeReference]

See TypedImperativeAggregate

Aggregation Buffer Schema

aggBufferSchema: StructType

Schema of an aggregation buffer with partial aggregate results

Used when:


inputAggBufferAttributes: Seq[AttributeReference]



foldable: Boolean

foldable is part of the Expression abstraction.

foldable is always false.

Converting to AggregateExpression

toAggregateExpression(): AggregateExpression // (1)
  isDistinct: Boolean,
  filter: Option[Expression] = None): AggregateExpression
  1. isDistinct flag is false

toAggregateExpression creates an AggregateExpression for this AggregateFunction and Complete mode.

import org.apache.spark.sql.functions.collect_list
val fn = collect_list("gid")

import org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression
val aggFn = fn.expr.asInstanceOf[AggregateExpression].aggregateFunction

scala> println(aggFn.numberedTreeString)
00 collect_list('gid, 0, 0)
01 +- 'gid

toAggregateExpression is used when: