UserDefinedAggregator¶

UserDefinedAggregator[IN, BUF, OUT] is a UserDefinedFunction that uses ScalaAggregator for execution.

UserDefinedAggregator is created using udaf standard function.

Creating Instance¶

UserDefinedAggregator takes the following to be created:

• Aggregator
• Encoder (of INs and assumed an ExpressionEncoder)
• Name
• nullable flag (default: true)
• deterministic flag (default: true)

UserDefinedAggregator is created using udaf standard function.

Creating Column (for Function Execution)¶

apply(
  exprs: Column*): Column

apply is part of the UserDefinedFunction abstraction.

apply creates a Column with an AggregateExpression with the following:

• ScalaAggregator aggregate function
• Complete aggregate mode
• isDistinct flag disabled (false)

Creating ScalaAggregator¶

scalaAggregator(
  exprs: Seq[Expression]): ScalaAggregator[IN, BUF, OUT]

scalaAggregator assumes the following are all ExpressionEncoders:

1. Input Encoder (of INs)
2. Buffer Encoder (of BUFs) of the Aggregator

In the end, scalaAggregator creates a ScalaAggregator.

scalaAggregator is used when:

• UDFRegistration is requested to register a UserDefinedAggregator
• UserDefinedAggregator is requested to create a Column (for execution)

Creates Named UserDefinedAggregator¶

withName(
  name: String): UserDefinedAggregator[IN, BUF, OUT]

withName is part of the UserDefinedFunction abstraction.

withName creates a copy of this UserDefinedAggregator to use the given name as the name.