Skip to content

LogicalPlanDistinctKeys Logical Operators

LogicalPlanDistinctKeys is an extension of the LogicalPlan abstraction for logical operators that know their distinct keys.

All logical operators are LogicalPlanDistinctKeys.

Distinct Keys

distinctKeys: Set[ExpressionSet]
Lazy Value

distinctKeys is a Scala lazy value to guarantee that the code to initialize it is executed once only (when accessed for the first time) and the computed value never changes afterwards.

Learn more in the Scala Language Specification.

distinctKeys uses DistinctKeyVisitor to determine the distinct keys of this logical operator when spark.sql.optimizer.propagateDistinctKeys.enabled configuration property is enabled.

Otherwise, distinctKeys is always empty.

distinctKeys is used when:

  • EliminateOuterJoin logical optimization is executed
  • EliminateDistinct logical optimization is executed
  • RemoveRedundantAggregates logical optimization is executed
  • JoinEstimation is requested to estimateInnerOuterJoin
  • SizeInBytesOnlyStatsPlanVisitor is requested to visitJoin