CodegenFallback Expressions¶

CodegenFallback is an extension of the Expression abstraction for Catalyst expressions that do not support Java code generation and fall back to interpreted mode (aka fallback mode).

CodegenFallback is used when:

CollapseCodegenStages physical optimization is executed (and enforce whole-stage codegen requirements for Catalyst expressions)
Generator expressions is requested to supportCodegen
EquivalentExpressions is requested to childrenToRecurse and commonChildrenToRecurse

Implementations¶

Generating Java Source Code¶

doGenCode(
  ctx: CodegenContext,
  ev: ExprCode): ExprCode

doGenCode is part of the Expression abstraction.

doGenCode requests the input CodegenContext to add itself to the references.

doGenCode walks down the expression tree to find Nondeterministic expressions. For every Nondeterministic expression, doGenCode does the following:

Requests the input CodegenContext to add it to the references
Requests the input CodegenContext to addPartitionInitializationStatement that is a Java code block as follows:
```
((Nondeterministic) references[[childIndex]])
  .initialize(partitionIndex);
```

In the end, doGenCode generates a plain Java source code block that is one of the following code blocks per the nullable flag. doGenCode copies the input ExprCode with the code block added (as the code property).

nullable enablednullable disabled

[placeHolder]
Object [objectTerm] = ((Expression) references[[idx]]).eval([input]);
boolean [isNull] = [objectTerm] == null;
[javaType] [value] = [defaultValue];
if (![isNull]) {
  [value] = ([boxedType]) [objectTerm];
}

[placeHolder]
Object [objectTerm] = ((Expression) references[[idx]]).eval([input]);
[javaType] [value] = ([boxedType]) [objectTerm];

Demo: CurrentTimestamp with nullable disabled¶

import org.apache.spark.sql.catalyst.expressions.CurrentTimestamp
val currTimestamp = CurrentTimestamp()

import org.apache.spark.sql.catalyst.expressions.codegen.CodegenFallback
assert(currTimestamp.isInstanceOf[CodegenFallback], "CurrentTimestamp should be a CodegenFallback")

assert(currTimestamp.nullable == false, "CurrentTimestamp should not be nullable")

import org.apache.spark.sql.catalyst.expressions.codegen.{CodegenContext, ExprCode}
val ctx = new CodegenContext
// doGenCode is used when Expression.genCode is executed
val ExprCode(code, _, _) = currTimestamp.genCode(ctx)

println(code)

Object obj_0 = ((Expression) references[0]).eval(null);
        long value_0 = (Long) obj_0;