Skip to content

CodegenFallback Expressions

CodegenFallback is an extension of the Expression abstraction for Catalyst expressions that do not support Java code generation and fall back to interpreted mode (aka fallback mode).

CodegenFallback is used when:


Generating Java Source Code

  ctx: CodegenContext,
  ev: ExprCode): ExprCode

doGenCode is part of the Expression abstraction.

doGenCode requests the input CodegenContext to add itself to the references.

doGenCode walks down the expression tree to find Nondeterministic expressions. For every Nondeterministic expression, doGenCode does the following:

  1. Requests the input CodegenContext to add it to the references

  2. Requests the input CodegenContext to addPartitionInitializationStatement that is a Java code block as follows:

    ((Nondeterministic) references[[childIndex]])

In the end, doGenCode generates a plain Java source code block that is one of the following code blocks per the nullable flag. doGenCode copies the input ExprCode with the code block added (as the code property).

Object [objectTerm] = ((Expression) references[[idx]]).eval([input]);
boolean [isNull] = [objectTerm] == null;
[javaType] [value] = [defaultValue];
if (![isNull]) {
  [value] = ([boxedType]) [objectTerm];
Object [objectTerm] = ((Expression) references[[idx]]).eval([input]);
[javaType] [value] = ([boxedType]) [objectTerm];

Demo: CurrentTimestamp with nullable disabled

import org.apache.spark.sql.catalyst.expressions.CurrentTimestamp
val currTimestamp = CurrentTimestamp()

import org.apache.spark.sql.catalyst.expressions.codegen.CodegenFallback
assert(currTimestamp.isInstanceOf[CodegenFallback], "CurrentTimestamp should be a CodegenFallback")

assert(currTimestamp.nullable == false, "CurrentTimestamp should not be nullable")
import org.apache.spark.sql.catalyst.expressions.codegen.{CodegenContext, ExprCode}
val ctx = new CodegenContext
// doGenCode is used when Expression.genCode is executed
val ExprCode(code, _, _) = currTimestamp.genCode(ctx)
Object obj_0 = ((Expression) references[0]).eval(null);
        long value_0 = (Long) obj_0;