CoalesceBucketsInJoin Physical Optimization¶

CoalesceBucketsInJoin is a physical query optimization (aka physical query preparation rule or simply preparation rule).

CollapseCodegenStages is a Catalyst rule for transforming physical query plans (Rule[SparkPlan]).

CoalesceBucketsInJoin is part of the preparations batch of physical query plan rules and is executed when QueryExecution is requested for the optimized physical query plan (in executedPlan phase of a query execution).

spark.sql.bucketing.coalesceBucketsInJoin.enabled¶

CoalesceBucketsInJoin uses the spark.sql.bucketing.coalesceBucketsInJoin.enabled configuration property.

Executing Rule¶

apply(
  plan: SparkPlan): SparkPlan

apply is part of the Rule abstraction.

apply is a noop with the spark.sql.bucketing.coalesceBucketsInJoin.enabled configuration property turned off.

apply uses ExtractJoinWithBuckets to match on BaseJoinExec physical operators.

apply...FIXME