PhysicalAggregation Scala Extractor¶
PhysicalAggregation
is a Scala extractor to destructure an Aggregate logical operator into a four-element tuple (ReturnType
) with the following elements:
- NamedExpressions of the grouping keys
- AggregateExpressions
- NamedExpressions of the result
- Child logical operator
ReturnType
(Seq[NamedExpression], Seq[AggregateExpression], Seq[NamedExpression], LogicalPlan)
Scala Extractor Object
Learn more in the Scala extractor objects.
Destructuring Aggregate Logical Operator¶
type ReturnType =
(Seq[NamedExpression], // Grouping Keys
Seq[AggregateExpression], // Aggregate Functions
Seq[NamedExpression], // Result
LogicalPlan) // Child
unapply(
a: Any): Option[ReturnType]
unapply
destructures an Aggregate logical operator into a four-element ReturnType tuple.
unapply
creates a EquivalentExpressions (to eliminate duplicate aggregate expressions and avoid evaluating them multiple times).
unapply
collects AggregateExpressions in the resultExpressions of the given Aggregate logical operator.
Some Other Magic
unapply
does some other magic but it does not look interesting, but the main idea should already be explained 😉
unapply
is used when:
StatefulAggregationStrategy
(Spark Structured Streaming) execution planning strategy is executed- Aggregation execution planning strategy is executed