Subexpression Elimination In Code-Generated Expression Evaluation (Common Expression Reuse)¶

Subexpression Elimination (aka Common Expression Reuse) is an optimization of a logical query plan that eliminates expressions in code-generated (non-interpreted) expression evaluation.

Subexpression Elimination is enabled by default. Use the internal <> configuration property control whether the feature is enabled (true) or not (false).

Subexpression Elimination is used (by means of SparkPlan.md#subexpressionEliminationEnabled[subexpressionEliminationEnabled] flag of SparkPlan) when the following physical operators are requested to execute (i.e. moving away from queries to an RDD of internal rows to describe a distributed computation):

• ProjectExec
• HashAggregateExec
• ObjectHashAggregateExec
• SortAggregateExec
• WindowExec (and creates a lookup table for WindowExpressions and factory functions for WindowFunctionFrame)

Subexpression Elimination is used when CodegenContext is requested for subexpressionElimination (when CodegenContext is requested to generateExpressions with subexpression elimination enabled).

spark.sql.subexpressionElimination.enabled Configuration Property¶

spark.sql.subexpressionElimination.enabled internal configuration property controls whether the subexpression elimination optimization is enabled or not.