PartitionPruning Logical Optimization¶
PartitionPruning is a logical optimization for Dynamic Partition Pruning.
apply( plan: LogicalPlan): LogicalPlan
Subquery operators that are
apply simply does nothing and gives it back unmodified.
apply does nothing when the spark.sql.optimizer.dynamicPartitionPruning.enabled configuration property is disabled (
For all other cases,
apply applies prune optimization.
apply is part of the Rule abstraction.
prune Internal Method¶
prune( plan: LogicalPlan): LogicalPlan
prune transforms up all logical operators in the given logical query plan.
prune transforms Join operators of the following "shape":
EqualTo join conditions
The join type is one of
More Work Needed
prune needs more love and would benefit from more insight on how it works.
prune is used when
PartitionPruning is executed.