Skip to content

AggregateFunction Expressions

AggregateFunction is an extension of the Expression abstraction for aggregate functions.

AggregateFunction can never be foldable.

Contract

Aggregation Buffer Attributes

aggBufferAttributes: Seq[AttributeReference]

See TypedImperativeAggregate

Aggregation Buffer Schema

aggBufferSchema: StructType

Schema of an aggregation buffer with partial aggregate results

Used when:

inputAggBufferAttributes

inputAggBufferAttributes: Seq[AttributeReference]

Implementations

foldable

Expression 
foldable: Boolean

foldable is part of the Expression abstraction.

foldable is always disabled (false).

Converting into AggregateExpression

toAggregateExpression(): AggregateExpression // (1)
toAggregateExpression(
  isDistinct: Boolean,
  filter: Option[Expression] = None): AggregateExpression
  1. isDistinct flag is false

toAggregateExpression creates an AggregateExpression for this AggregateFunction (with Complete mode).

import org.apache.spark.sql.functions.collect_list
val fn = collect_list("gid")

import org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression
val aggFn = fn.expr.asInstanceOf[AggregateExpression].aggregateFunction

scala> println(aggFn.numberedTreeString)
00 collect_list('gid, 0, 0)
01 +- 'gid

toAggregateExpression is used when: