Offset Logical Operator¶

Offset is an OrderPreservingUnaryNode that can skip the specified number of rows (offset) from the beginning of the output of the child logical operator.

Creating Instance¶

Offset takes the following to be created:

• Offset Expression
• Child Logical operator

Offset is created when:

• AstBuilder is requested to withQueryResultClauses
• Dataset.offset operator is used

Catalyst DSL¶

Catalyst DSL comes with offset operator to create an Offset.

offset(
  offsetExpr: Expression): LogicalPlan

Logical Optimizations¶

Offset is optimized using the following logical optimizations:

• EliminateOffsets
• LimitPushDown
• V2ScanRelationPushDown

Physical Planning¶

Offset is planned as either GlobalLimitExec or CollectLimitExec physical operators.

maxRows¶

maxRows: Option[Long]

maxRows...FIXME