Skip to content

RewriteCorrelatedScalarSubquery Logical Optimization

RewriteCorrelatedScalarSubquery is a base logical optimization that <> with the following operators:


RewriteCorrelatedScalarSubquery is part of the Operator Optimization before Inferring Filters fixed-point batch in the standard batches of the Logical Optimizer.

RewriteCorrelatedScalarSubquery is simply a <> for transforming <>, i.e. Rule[LogicalPlan].

[source, scala]

import org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery

// FIXME // Demo: Filter + Aggregate // Demo: Filter + UnaryNode

val plan = ??? val optimizedPlan = RewriteCorrelatedScalarSubquery(plan)

=== [[evalExpr]] evalExpr Internal Method

[source, scala]

evalExpr(expr: Expression, bindings: Map[ExprId, Option[Any]]) : Option[Any]


NOTE: evalExpr is used exclusively when RewriteCorrelatedScalarSubquery is...FIXME

=== [[evalAggOnZeroTups]] evalAggOnZeroTups Internal Method

[source, scala]

evalAggOnZeroTups(expr: Expression) : Option[Any]


NOTE: evalAggOnZeroTups is used exclusively when RewriteCorrelatedScalarSubquery is...FIXME

=== [[evalSubqueryOnZeroTups]] evalSubqueryOnZeroTups Internal Method

[source, scala]

evalSubqueryOnZeroTups(plan: LogicalPlan) : Option[Any]


NOTE: evalSubqueryOnZeroTups is used exclusively when RewriteCorrelatedScalarSubquery is requsted to <>.

=== [[constructLeftJoins]] constructLeftJoins Internal Method

[source, scala]

constructLeftJoins( child: LogicalPlan, subqueries: ArrayBuffer[ScalarSubquery]): LogicalPlan


NOTE: constructLeftJoins is used exclusively when RewriteCorrelatedScalarSubquery logical optimization is <> (i.e. applied to <>, <> or Filter logical operators with correlated scalar subqueries)

=== [[extractCorrelatedScalarSubqueries]] Extracting ScalarSubquery Expressions with Children -- extractCorrelatedScalarSubqueries Internal Method

[source, scala]

extractCorrelatedScalarSubqueriesE <: Expression: E

extractCorrelatedScalarSubqueries finds all ScalarSubquery expressions with at least one child in the input expression and adds them to the input subqueries collection.

extractCorrelatedScalarSubqueries traverses the input expression down (the expression tree) and, every time a ScalarSubquery with at least one child is found, returns the head of the output attributes of the subquery plan.

In the end, extractCorrelatedScalarSubqueries returns the rewritten expression.

NOTE: extractCorrelatedScalarSubqueries uses[scala.collection.mutable.ArrayBuffer] and mutates an instance inside (i.e. adds ScalarSubquery expressions) that makes for two output values, i.e. the rewritten expression and the ScalarSubquery expressions.

NOTE: extractCorrelatedScalarSubqueries is used exclusively when RewriteCorrelatedScalarSubquery is <> (i.e. applied to a[logical plan]).