PropagateEmptyRelation Logical Optimization¶

PropagateEmptyRelation is a base logical optimization that <>, e.g. <> or <>.

PropagateEmptyRelation is part of the LocalRelation fixed-point batch in the standard batches of the Logical Optimizer.

PropagateEmptyRelation is simply a <> for transforming <>, i.e. Rule[LogicalPlan].

=== [[explode]] Explode

[source, scala]¶

scala> val emp = spark.emptyDataset[Seq[String]] emp: org.apache.spark.sql.Dataset[Seq[String]] = [value: array]

scala> emp.select(explode($"value")).show +---+ |col| +---+ +---+

scala> emp.select(explode($"value")).explain(true) == Parsed Logical Plan == 'Project [explode('value) AS List()] +- LocalRelation , [value#77]

== Analyzed Logical Plan == col: string Project [col#89] +- Generate explode(value#77), false, false, [col#89] +- LocalRelation , [value#77]

== Optimized Logical Plan == LocalRelation , [col#89]

== Physical Plan == LocalTableScan , [col#89]

=== [[join]] Join

[source, scala]¶

scala> spark.emptyDataset[Int].join(spark.range(1)).explain(extended = true) ... TRACE SparkOptimizer: === Applying Rule org.apache.spark.sql.catalyst.optimizer.PropagateEmptyRelation === !Join Inner LocalRelation , [value#40, id#42L] !:- LocalRelation , [value#40] !+- Range (0, 1, step=1, splits=Some(8))

TRACE SparkOptimizer: Fixed point reached for batch LocalRelation after 2 iterations. DEBUG SparkOptimizer: === Result of Batch LocalRelation === !Join Inner LocalRelation , [value#40, id#42L] !:- LocalRelation , [value#40] !+- Range (0, 1, step=1, splits=Some(8)) ... == Parsed Logical Plan == Join Inner :- LocalRelation , [value#40] +- Range (0, 1, step=1, splits=Some(8))

== Analyzed Logical Plan == value: int, id: bigint Join Inner :- LocalRelation , [value#40] +- Range (0, 1, step=1, splits=Some(8))

== Optimized Logical Plan == LocalRelation , [value#40, id#42L]

== Physical Plan == LocalTableScan , [value#40, id#42L]

Executing Rule¶

apply(plan: LogicalPlan): LogicalPlan

apply...FIXME

apply is part of the Rule abstraction.