Skip to content

Intersect Logical Operator

Intersect is a SetOperation binary logical operator that represents the following high-level operators in a logical plan:

Intersect is replaced at logical optimization phase (based on isAll flag):

Logical Operators Logical Optimization isAll
Left Semi Join ReplaceIntersectWithSemiJoin disabled
Generate over Aggregate over Union RewriteIntersectAll enabled
Spark Structured Streaming Unsupported

Intersect is not supported on streaming DataFrames/Datasets (that is enforced by UnsupportedOperationChecker at QueryExecution).

Creating Instance

Intersect takes the following to be created:

Intersect is created when:

Catalyst DSL

Catalyst DSL comes with intersect operator to create an Intersect operator.

  otherPlan: LogicalPlan,
  isAll: Boolean): LogicalPlan