RepartitionByExpression Logical Operator¶
RepartitionByExpression is a concrete RepartitionOperation.
RepartitionByExpression is also called distribute operator.
RepartitionByExpression takes the following to be created:
RepartitionByExpression is created when:
- Dataset.repartition and Dataset.repartitionByRange operators
REPARTITION_BY_RANGEhints (via ResolveCoalesceHints logical analysis rule)
CLUSTER BYSQL clauses (via SparkSqlAstBuilder)
RepartitionByExpression is planned to ShuffleExchangeExec physical operator.
Maximum Number of Rows¶
maxRows is part of the LogicalPlan abstraction.
shuffle is always
shuffle is part of the RepartitionOperation abstraction.