LogicalPlanDistinctKeys Logical Operators¶

LogicalPlanDistinctKeys is an extension of the LogicalPlan abstraction for logical operators that know their distinct keys.

All logical operators are LogicalPlanDistinctKeys.

Distinct Keys¶

distinctKeys: Set[ExpressionSet]

Lazy Value

distinctKeys is a Scala lazy value to guarantee that the code to initialize it is executed once only (when accessed for the first time) and the computed value never changes afterwards.

Learn more in the Scala Language Specification.

distinctKeys uses DistinctKeyVisitor to determine the distinct keys of this logical operator when spark.sql.optimizer.propagateDistinctKeys.enabled configuration property is enabled.

Otherwise, distinctKeys is always empty.

distinctKeys is used when:

EliminateOuterJoin logical optimization is executed
EliminateDistinct logical optimization is executed
RemoveRedundantAggregates logical optimization is executed
JoinEstimation is requested to estimateInnerOuterJoin
SizeInBytesOnlyStatsPlanVisitor is requested to visitJoin