ReuseAdaptiveSubquery Physical Optimization¶

ReuseAdaptiveSubquery is a physical query plan optimization in Adaptive Query Execution.

ReuseAdaptiveSubquery is a Catalyst Rule for transforming physical plans (Rule[SparkPlan]).

Creating Instance¶

ReuseAdaptiveSubquery takes the following to be created:

• Subquery Cache

ReuseAdaptiveSubquery is created when:

• AdaptiveSparkPlanExec leaf physical operator is requested for the adaptive optimizations

Executing Rule¶

apply(
  plan: SparkPlan): SparkPlan

apply is disabled (and returns the given SparkPlan) when the spark.sql.execution.reuseSubquery configuration property is false.

apply requests the given SparkPlan to transformAllExpressionsWithPruning with tree nodes with PLAN_EXPRESSION tree pattern:

• For a ExecSubqueryExpression expression, apply replaces the plan with a new ReusedSubqueryExec physical operator with a cached plan if found in the cache.

apply is part of the Rule abstraction.