Skip to content

Union Logical Operator

Union is a logical operator that represents the following high-level operators in a logical plan:

Union is resolved using ResolveUnion logical analysis rule.

Union is resolved into UnionExec physical operator by BasicOperators execution planning strategy.

Creating Instance

Union takes the following to be created:

  • Child LogicalPlans
  • byName flag (default: false)
  • allowMissingCol flag (default: false)


allowMissingCol can be true only with byName being true.

Union is created (possibly using apply utility) when:

Creating Union

  left: LogicalPlan,
  right: LogicalPlan): Union

apply creates a Union logical operator (with the left and right plans as the children operators).

apply is used when:

  • ResolveUnion logical resolution rule is executed
  • RewriteUpdateTable is requested to buildReplaceDataWithUnionPlan
  • RewriteExceptAll logical optimization is executed
  • RewriteIntersectAll logical optimization is executed
  • AstBuilder is requested to parse UNION SQL statement
  • Dataset.union operator is used

Maximum Number of Records

maxRows: Option[Long]

maxRows is part of the LogicalPlan abstraction.

maxRows is the total of the maxRows of all the children.

Node Patterns

nodePatterns: Seq[TreePattern]

nodePatterns is part of the TreeNode abstraction.

nodePatterns is UNION.

Metadata Output Attributes

metadataOutput: Seq[Attribute]

metadataOutput is part of the LogicalPlan abstraction.

metadataOutput is empty.

Catalyst DSL

Catalyst DSL comes with union operator to create an Union operator.

  otherPlan: LogicalPlan): LogicalPlan

Logical Optimizations