Subexpression Elimination In Code-Generated Expression Evaluation (Common Expression Reuse)¶

Subexpression Elimination (aka Common Expression Reuse) is an optimization of a logical query plan that eliminates expressions in code-generated (non-interpreted) expression evaluation.

Subexpression Elimination is enabled by default. Use the internal <> configuration property control whether the feature is enabled (true) or not (false).

Subexpression Elimination is used (by means of SparkPlan.md#subexpressionEliminationEnabled[subexpressionEliminationEnabled] flag of SparkPlan) when the following physical operators are requested to execute (i.e. moving away from queries to an RDD of internal rows to describe a distributed computation):

• ProjectExec

• HashAggregateExec

• ObjectHashAggregateExec

• SortAggregateExec

• WindowExec (and creates a lookup table for WindowExpressions and factory functions for WindowFunctionFrame)

Internally, subexpression elimination happens when CodegenContext is requested for subexpressionElimination (when CodegenContext is requested to <> with <> enabled).

spark.sql.subexpressionElimination.enabled Configuration Property¶

spark.sql.subexpressionElimination.enabled internal configuration property controls whether the subexpression elimination optimization is enabled or not.