PullupCorrelatedPredicates Logical Optimization¶

PullupCorrelatedPredicates is a base logical optimization that <> with the following operators:

. Filter operators with an Aggregate.md[Aggregate] child operator

. UnaryNode operators

PullupCorrelatedPredicates is part of the Pullup Correlated Expressions once-executed batch in the standard batches of the Logical Optimizer.

PullupCorrelatedPredicates is simply a <> for transforming <>, i.e. Rule[LogicalPlan].

[source, scala]¶

import org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates

// FIXME // Demo: Filter + Aggregate // Demo: Filter + UnaryNode

val plan = ??? val optimizedPlan = PullupCorrelatedPredicates(plan)

PullupCorrelatedPredicates uses PredicateHelper.

=== [[pullOutCorrelatedPredicates]] pullOutCorrelatedPredicates Internal Method

[source, scala]¶

pullOutCorrelatedPredicates( sub: LogicalPlan, outer: Seq[LogicalPlan]): (LogicalPlan, Seq[Expression])

pullOutCorrelatedPredicates...FIXME

NOTE: pullOutCorrelatedPredicates is used exclusively when PullupCorrelatedPredicates is requested to <>.

=== [[rewriteSubQueries]] rewriteSubQueries Internal Method

[source, scala]¶

rewriteSubQueries(plan: LogicalPlan, outerPlans: Seq[LogicalPlan]): LogicalPlan¶

rewriteSubQueries...FIXME

NOTE: rewriteSubQueries is used exclusively when PullupCorrelatedPredicates is <> (i.e. applied to a spark-sql-LogicalPlan.md[logical plan]).

=== [[apply]] Executing Rule -- apply Method

[source, scala]¶

apply(plan: LogicalPlan): LogicalPlan¶

NOTE: apply is part of the <> to execute (apply) a rule on a TreeNode (e.g. <>).

apply transforms the input spark-sql-LogicalPlan.md[logical plan] as follows:

. For Filter operators with an Aggregate.md[Aggregate] child operator, apply <> with the Filter and the Aggregate and its Aggregate.md#child[child] as the outer plans

. For spark-sql-LogicalPlan.md#UnaryNode[UnaryNode] operators, apply <> with the operator and its children as the outer plans