Subexpression Elimination In Code-Generated Expression Evaluation (Common Expression Reuse)¶
Subexpression Elimination (aka Common Expression Reuse) is an optimization of a logical query plan that eliminates expressions in code-generated (non-interpreted) expression evaluation.
Subexpression Elimination is enabled by default. Use the internal <
true) or not (
Subexpression Elimination is used (by means of SparkPlan.md#subexpressionEliminationEnabled[subexpressionEliminationEnabled] flag of
SparkPlan) when the following physical operators are requested to execute (i.e. moving away from queries to an RDD of internal rows to describe a distributed computation):
WindowExec (and creates a lookup table for WindowExpressions and factory functions for WindowFunctionFrame)
Internally, subexpression elimination happens when
CodegenContext is requested for subexpressionElimination (when
CodegenContext is requested to <
spark.sql.subexpressionElimination.enabled Configuration Property¶
spark.sql.subexpressionElimination.enabled internal configuration property controls whether the subexpression elimination optimization is enabled or not.