ResolveJoinStrategyHints Logical Resolution Rule¶

ResolveJoinStrategyHints is a logical resolution rule to resolve UnresolvedHint logical operators with JoinStrategyHints.

ResolveJoinStrategyHints is a Catalyst rule for transforming logical plans (Rule[LogicalPlan]).

ResolveJoinStrategyHints is part of Hints batch of rules of Logical Analyzer.

Creating Instance¶

ResolveJoinStrategyHints takes the following to be created:

SQLConf

ResolveJoinStrategyHints is created when:

Logical Analyzer is requested for the batches of rules

Hint Names¶

ResolveJoinStrategyHints takes the hintAliases of the strategies when created.

The hint aliases are the only hints (of UnresolvedHints) that are going to be resolved when ResolveJoinStrategyHints is executed.

Executing Rule¶

apply(
  plan: LogicalPlan): LogicalPlan

apply works with LogicalPlans that contain the UNRESOLVED_HINT tree pattern (that happens to be UnresolvedHints only).

apply traverses the given logical query plan to find UnresolvedHints with names that are among the supported hint names.

For UnresolvedHints with no parameters, apply creates a ResolvedHint (with a HintInfo with the corresponding JoinStrategyHint).

For UnresolvedHints with parameters, apply accepts two types of parameters:

Table Name (as a String)
Table Identifier (as a UnresolvedAttribute)

apply applyJoinStrategyHint to create a ResolvedHint.

apply is part of the Rule abstraction.

applyJoinStrategyHint¶

applyJoinStrategyHint(
  plan: LogicalPlan,
  relationsInHint: Set[Seq[String]],
  relationsInHintWithMatch: mutable.HashSet[Seq[String]],
  hintName: String): LogicalPlan

applyJoinStrategyHint...FIXME

Demo¶

Fixme

Review the example to use ResolveJoinStrategyHints and other hints

// Use Catalyst DSL to create a logical plan
import org.apache.spark.sql.catalyst.dsl.plans._
val plan = table("t1").join(table("t2")).hint(name = "broadcast", "t1", "table2")

scala> println(plan.numberedTreeString)
00 'UnresolvedHint broadcast, [t1, table2]
01 +- 'Join Inner
02    :- 'UnresolvedRelation [t1], [], false
03    +- 'UnresolvedRelation [t2], [], false

import org.apache.spark.sql.catalyst.analysis.ResolveHints.ResolveJoinStrategyHints
val analyzedPlan = ResolveJoinStrategyHints(plan)

scala> println(analyzedPlan.numberedTreeString)
00 'Join Inner
01 :- 'ResolvedHint (strategy=broadcast)
02 :  +- 'UnresolvedRelation [t1], [], false
03 +- 'UnresolvedRelation [t2], [], false