ResolveSubquery Logical Resolution Rule¶

ResolveSubquery is a logical resolution rule that resolves SubqueryExpressions in the following logical operators:

ResolveSubquery resolves subqueries (logical query plans) in the following SubqueryExpressions:

ResolveSubquery is part of Resolution rule batch of the Logical Analyzer.

ResolveSubquery is a Catalyst rule for transforming logical plans (Rule[LogicalPlan]).

Example¶

// Use Catalyst DSL
import org.apache.spark.sql.catalyst.expressions._
val a = 'a.int

import org.apache.spark.sql.catalyst.plans.logical.LocalRelation
val rel = LocalRelation(a)

import org.apache.spark.sql.catalyst.expressions.Literal
val list = Seq[Literal](1)

// FIXME Use a correct query to demo ResolveSubquery
import org.apache.spark.sql.catalyst.plans.logical.Filter
import org.apache.spark.sql.catalyst.expressions.In
val plan = Filter(condition = In(value = a, list), child = rel)

scala> println(plan.numberedTreeString)
00 Filter a#9 IN (1)
01 +- LocalRelation <empty>, [a#9]

import spark.sessionState.analyzer.ResolveSubquery
val analyzedPlan = ResolveSubquery(plan)
scala> println(analyzedPlan.numberedTreeString)
00 Filter a#9 IN (1)
01 +- LocalRelation <empty>, [a#9]

Executing Rule¶

apply(
  plan: LogicalPlan): LogicalPlan

apply resolves SubqueryExpressions in the following logical operators in the given logical plan:

apply resolves logical operators that have their children all resolved already.

apply is part of Rule abstraction.

Resolving SubqueryExpressions¶

resolveSubQueries(
  plan: LogicalPlan,
  plans: Seq[LogicalPlan]): LogicalPlan

resolveSubQueries resolves subquery plans in the following SubqueryExpression expressions (by transforming expressions down the operator tree) in the given logical plan:

resolveSubQueries is used when ResolveSubquery is executed.

Resolving Subquery Plan¶

resolveSubQuery(
  e: SubqueryExpression,
  plans: Seq[LogicalPlan])(
  f: (LogicalPlan, Seq[Expression]) => SubqueryExpression): SubqueryExpression

resolveSubQuery resolves outer references in the logical plan of the given SubqueryExpression (and all sub-SubqueryExpressions).

In the end, resolveSubQuery executes the given f function with the logical plan (of the SubqueryExpression) and all OuterReference leaf expressions when the logical plan has been fully resolved. Otherwise, resolveSubQuery requests the SubqueryExpression to withNewPlan.

resolveSubQuery is used when ResolveSubquery is requested to resolve subquery expressions.

Resolving Outer References (in Subquery Plan)¶

resolveOuterReferences(
  plan: LogicalPlan,
  outer: LogicalPlan): LogicalPlan

resolveOuterReferences uses the outer logical plan to resolve UnresolvedAttribute expressions in the plan logical operator.

resolveOuterReferences translates resolved NamedExpression expressions to OuterReference leaf expressions.

resolveOuterReferences is used when ResolveSubquery is requested to resolve a SubqueryExpression.