Skip to content

PlanAdaptiveDynamicPruningFilters Physical Optimization

PlanAdaptiveDynamicPruningFilters is a physical optimization in Adaptive Query Execution.

PlanAdaptiveDynamicPruningFilters is a Catalyst Rule for transforming physical plans (Rule[SparkPlan]).

Creating Instance

PlanAdaptiveDynamicPruningFilters takes the following to be created:

PlanAdaptiveDynamicPruningFilters is created when:

Executing Rule

  plan: SparkPlan): SparkPlan

apply is part of the Rule abstraction.

apply is disabled (and simply returns the given SparkPlan unchanged) when the spark.sql.optimizer.dynamicPartitionPruning.enabled is disabled.

apply requests the given SparkPlan to transformAllExpressionsWithPruning in QueryPlans with the DYNAMIC_PRUNING_EXPRESSION and IN_SUBQUERY_EXEC tree patterns:

In the end, apply creates a new DynamicPruningExpression unary expression (with a InSubqueryExec or TrueLiteral).