UnsafeProjection¶

UnsafeProjection is an extension of the Projection abstraction for projection functions from InternalRows to (produce) UnsafeRows.

UnsafeProjection: InternalRow => UnsafeRow

UnsafeProjection is created for code generated and interpreted code evaluation paths (using UnsafeProjection factory object).

Contract¶

Encoding InternalRow as UnsafeRow¶

apply(
  row: InternalRow): UnsafeRow

Encodes the given InternalRow to an UnsafeRow

Implementations¶

• InterpretedUnsafeProjection

CodeGeneratorWithInterpretedFallback¶

UnsafeProjection factory object is a CodeGeneratorWithInterpretedFallback of UnsafeProjections (based on Expressions).

CodeGeneratorWithInterpretedFallback[Seq[Expression], UnsafeProjection]

createCodeGeneratedObject¶

createCodeGeneratedObject(
  in: Seq[Expression]): UnsafeProjection

createCodeGeneratedObject generates an UnsafeProjection for the given Expressions (possibly with Subexpression Elimination based on spark.sql.subexpressionElimination.enabled configuration property).

createInterpretedObject¶

createInterpretedObject(
  in: Seq[Expression]): UnsafeProjection

createInterpretedObject creates an UnsafeProjection for the given Expressions.

Creating UnsafeProjection¶

create(
  fields: Array[DataType]): UnsafeProjection
create(
  expr: Expression): UnsafeProjection
create(
  exprs: Seq[Expression]): UnsafeProjection // (1)!
create(
  exprs: Seq[Expression],
  inputSchema: Seq[Attribute]): UnsafeProjection
create(
  schema: StructType): UnsafeProjection

1. The main create that the other variants depend on

create creates an UnsafeProjection for the given Expressions.

create is used when:

• ExpressionEncoder is requested to create a serializer
• V2Aggregator is requested for an inputProjection
• SerializeFromObjectExec physical operator is executed
• SortExec physical operator is requested to create an UnsafeExternalRowSorter
• HashAggregateExec physical operator is requested to create a hash map and getEmptyAggregationBuffer
• AggregationIterator is created and requested to generateResultProjection
• TungstenAggregationIterator is requested to create an aggregation buffer
• ScalaAggregator is requested for the inputProjection
• others