Subexpression Elimination In Code-Generated Expression Evaluation (Common Expression Reuse)¶
Subexpression Elimination (aka Common Expression Reuse) is an optimization of a logical query plan that eliminates expressions in code-generated (non-interpreted) expression evaluation.
Subexpression Elimination is enabled by default. Use the internal <true
) or not (false
).
Subexpression Elimination is used (by means of SparkPlan.md#subexpressionEliminationEnabled[subexpressionEliminationEnabled] flag of SparkPlan
) when the following physical operators are requested to execute (i.e. moving away from queries to an RDD of internal rows to describe a distributed computation):
-
WindowExec (and creates a lookup table for WindowExpressions and factory functions for WindowFunctionFrame)
Internally, subexpression elimination happens when CodegenContext
is requested for subexpressionElimination (when CodegenContext
is requested to <
spark.sql.subexpressionElimination.enabled Configuration Property¶
spark.sql.subexpressionElimination.enabled internal configuration property controls whether the subexpression elimination optimization is enabled or not.