GlobalLimit Logical Operator¶

GlobalLimit is an order-preserving unary logical operator.

Creating Instance¶

GlobalLimit takes the following to be created:

• Limit Expression
• Child query plan

GlobalLimit is created using Limit.apply utility.

Query Planning¶

GlobalLimit is planned to GlobalLimitExec physical operator (when BasicOperators execution planning strategy is executed).

Query Physical Optimization¶

• CombineLimits physical optimization

Creating GlobalLimit¶

apply(
  limitExpr: Expression,
  child: LogicalPlan): UnaryNode

apply creates a GlobalLimit for the given limitExpr expression and a LocalLimit logical operator.

apply is used when:

• AstBuilder is requested to withQueryResultClauses and withSample
• Dataset.limit operator is used
• RewriteNonCorrelatedExists logical optimization is executed
• CombineLimits physical optimization is executed
• Catalyst DSL's limit operator is used

Example¶

val q = spark.range(10).limit(3)
scala> q.explain(extended = true)
== Parsed Logical Plan ==
GlobalLimit 3
+- LocalLimit 3
   +- Range (0, 10, step=1, splits=Some(16))

== Analyzed Logical Plan ==
id: bigint
GlobalLimit 3
+- LocalLimit 3
   +- Range (0, 10, step=1, splits=Some(16))

== Optimized Logical Plan ==
GlobalLimit 3
+- LocalLimit 3
   +- Range (0, 10, step=1, splits=Some(16))

== Physical Plan ==
CollectLimit 3
+- *(1) Range (0, 10, step=1, splits=16)