ReuseExchangeAndSubquery Physical Optimization¶

ReuseExchangeAndSubquery is a physical query optimization.

ReuseExchangeAndSubquery is part of the preparations batch of physical query plan rules and is executed when QueryExecution is requested for the optimized physical query plan.

ReuseExchangeAndSubquery is a Catalyst rule for transforming physical query plans (Rule[SparkPlan]).

Executing Rule¶

apply(
  plan: SparkPlan): SparkPlan

apply is a noop (and simply returns the given SparkPlan unchanged) when neither the spark.sql.exchange.reuse nor the spark.sql.execution.reuseSubquery are enabled.

apply requests the SparkPlan to transformUpWithPruning any physical operator with EXCHANGE or PLAN_EXPRESSION tree pattern:

• For an Exchange and the spark.sql.exchange.reuse enabled, apply may create a ReusedExchangeExec if there is a cached (found earlier) exchange.

• For other physical operators, apply requests the current SparkPlan to transformExpressionsUpWithPruning any physical operator with PLAN_EXPRESSION tree pattern and may create a ReusedSubqueryExec for a ExecSubqueryExpression if there is a cached (found earlier) subquery and the spark.sql.execution.reuseSubquery is enabled

apply is part of the Rule abstraction.