UserDefinedAggregator¶
UserDefinedAggregator[IN, BUF, OUT]
is a UserDefinedFunction.
Creating Instance¶
UserDefinedAggregator
takes the following to be created:
- Aggregator[IN, BUF, OUT]
- Encoder[IN] (assumed a ExpressionEncoder)
- Name (optional)
-
nullable
flag (default:true
) -
deterministic
flag (default:true
)
UserDefinedAggregator
is created using udaf standard function.
Creating Column for UDAF Execution¶
apply(
exprs: Column*): Column
apply
creates a Column with an AggregateExpression with the following:
- ScalaAggregator aggregate function
Complete
aggregate modeisDistinct
flag disabled (false
)
apply
is part of the UserDefinedFunction abstraction.
Creating ScalaAggregator¶
scalaAggregator(
exprs: Seq[Expression]): ScalaAggregator[IN, BUF, OUT]
scalaAggregator
assumes the following are all ExpressionEncoders:
In the end, scalaAggregator
creates a ScalaAggregator.
scalaAggregator
is used when:
UDFRegistration
is requested to register a UserDefinedAggregatorUserDefinedAggregator
is requested to create a Column (for UDAF execution)