Repartition Logical Operator¶
Repartition is a concrete RepartitionOperation.
Repartition takes the following to be created:
Repartition is created for the following:
- Dataset.coalesce and Dataset.repartition operators (with shuffle disabled and enabled, respectively)
REPARTITIONhints (via ResolveCoalesceHints logical analysis rule, with shuffle disabled and enabled, respectively)
Repartition is planned to the following physical operators based on shuffle flag:
Catalyst DSL defines the following operators to create a
Repartition logical operator:
partitioning requires that the shuffle flag is enabled or throws an exception:
Partitioning can only be used in shuffle.
partitioning is part of the RepartitionOperation abstraction.