Skip to content

PullupCorrelatedPredicates Logical Optimization

PullupCorrelatedPredicates is a base logical optimization that <> with the following operators:

. Filter operators with an[Aggregate] child operator

. UnaryNode operators

PullupCorrelatedPredicates is part of the Pullup Correlated Expressions once-executed batch in the standard batches of the Logical Optimizer.

PullupCorrelatedPredicates is simply a <> for transforming <>, i.e. Rule[LogicalPlan].

[source, scala]

import org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates

// FIXME // Demo: Filter + Aggregate // Demo: Filter + UnaryNode

val plan = ??? val optimizedPlan = PullupCorrelatedPredicates(plan)

PullupCorrelatedPredicates uses PredicateHelper.

=== [[pullOutCorrelatedPredicates]] pullOutCorrelatedPredicates Internal Method

[source, scala]

pullOutCorrelatedPredicates( sub: LogicalPlan, outer: Seq[LogicalPlan]): (LogicalPlan, Seq[Expression])


NOTE: pullOutCorrelatedPredicates is used exclusively when PullupCorrelatedPredicates is requested to <>.

=== [[rewriteSubQueries]] rewriteSubQueries Internal Method

[source, scala]

rewriteSubQueries(plan: LogicalPlan, outerPlans: Seq[LogicalPlan]): LogicalPlan


NOTE: rewriteSubQueries is used exclusively when PullupCorrelatedPredicates is <> (i.e. applied to a[logical plan]).

=== [[apply]] Executing Rule -- apply Method

[source, scala]

apply(plan: LogicalPlan): LogicalPlan

NOTE: apply is part of the <> to execute (apply) a rule on a TreeNode (e.g. <>).

apply transforms the input[logical plan] as follows:

. For Filter operators with an[Aggregate] child operator, apply <> with the Filter and the Aggregate and its[child] as the outer plans

. For[UnaryNode] operators, apply <> with the operator and its children as the outer plans