Skip to content


UserDefinedAggregator[IN, BUF, OUT] is a UserDefinedFunction.

Creating Instance

UserDefinedAggregator takes the following to be created:

UserDefinedAggregator is created using udaf standard function.

Creating Column for UDAF Execution

  exprs: Column*): Column

apply creates a Column with an AggregateExpression with the following:

  • ScalaAggregator aggregate function
  • Complete aggregate mode
  • isDistinct flag disabled (false)

apply is part of the UserDefinedFunction abstraction.

Creating ScalaAggregator

  exprs: Seq[Expression]): ScalaAggregator[IN, BUF, OUT]

scalaAggregator assumes the following are all ExpressionEncoders:

  1. Input Encoder
  2. Buffer Encoder of the Aggregator

In the end, scalaAggregator creates a ScalaAggregator.

scalaAggregator is used when: