AQEShuffleReadRule Physical Optimization Rules¶

AQEShuffleReadRule is an extension of the Rule abstraction for physical optimization rules (Rule[SparkPlan]) that supportedShuffleOrigins.

Contract¶

Supported ShuffleOrigins¶

supportedShuffleOrigins: Seq[ShuffleOrigin]

Supported ShuffleOrigins

See:

• CoalesceShufflePartitions
• OptimizeShuffleWithLocalRead
• OptimizeSkewInRebalancePartitions

Used when:

• AdaptiveSparkPlanExec physical operator is requested for the final physical query plan
• others (FIXME: perhaps not as important?)

Implementations¶

• CoalesceShufflePartitions
• OptimizeShuffleWithLocalRead
• OptimizeSkewInRebalancePartitions