ResolveJoinStrategyHints Logical Resolution Rule¶
ResolveJoinStrategyHints is a logical resolution rule to resolve UnresolvedHint logical operators with JoinStrategyHints.
ResolveJoinStrategyHints is a Catalyst rule for transforming logical plans (
ResolveJoinStrategyHints is part of Hints batch of rules of Logical Analyzer.
ResolveJoinStrategyHints takes the following to be created:
ResolveJoinStrategyHints is created when:
- Logical Analyzer is requested for the batches of rules
ResolveJoinStrategyHints takes the hintAliases of the strategies when created.
The hint aliases are the only hints (of UnresolvedHints) that are going to be resolved when
ResolveJoinStrategyHints is executed.
apply( plan: LogicalPlan): LogicalPlan
apply works with LogicalPlans that contain the UNRESOLVED_HINT tree pattern (that happens to be UnresolvedHints only).
apply traverses the given logical query plan to find UnresolvedHints with names that are among the supported hint names.
UnresolvedHints with no parameters,
apply creates a ResolvedHint (with a HintInfo with the corresponding JoinStrategyHint).
UnresolvedHints with parameters,
apply accepts two types of parameters:
- Table Name (as a
- Table Identifier (as a UnresolvedAttribute)
apply applyJoinStrategyHint to create a ResolvedHint.
apply is part of the Rule abstraction.
applyJoinStrategyHint( plan: LogicalPlan, relationsInHint: Set[Seq[String]], relationsInHintWithMatch: mutable.HashSet[Seq[String]], hintName: String): LogicalPlan
Review the example to use
ResolveJoinStrategyHints and other hints
// Use Catalyst DSL to create a logical plan import org.apache.spark.sql.catalyst.dsl.plans._ val plan = table("t1").join(table("t2")).hint(name = "broadcast", "t1", "table2")
scala> println(plan.numberedTreeString) 00 'UnresolvedHint broadcast, [t1, table2] 01 +- 'Join Inner 02 :- 'UnresolvedRelation [t1], , false 03 +- 'UnresolvedRelation [t2], , false
import org.apache.spark.sql.catalyst.analysis.ResolveHints.ResolveJoinStrategyHints val analyzedPlan = ResolveJoinStrategyHints(plan)
scala> println(analyzedPlan.numberedTreeString) 00 'Join Inner 01 :- 'ResolvedHint (strategy=broadcast) 02 : +- 'UnresolvedRelation [t1], , false 03 +- 'UnresolvedRelation [t2], , false