Skip to content

Generator Expressions

Generator is an extension of the Expression abstraction for generator expressions that can produce multiple rows for a single input row.

The execution of Generator is managed by GenerateExec unary physical operator.

Note

Generator corresponds to LATERAL VIEW in SQL.

Contract (Subset)

Interpreted Expression Evaluation

eval(
  input: InternalRow): TraversableOnce[InternalRow]

Evaluates the given InternalRow to produce zero, one or more InternalRows

Return Type

eval is part of the Expression abstraction.

This eval enforces that Generators produce a collection of InternalRows (not any other value as by non-generator expressions).

Implementations

  • CollectionGenerator
  • GeneratorOuter
  • HiveGenericUDTF
  • JsonTuple
  • ReplicateRows
  • SQLKeywords
  • Stack
  • UnevaluableGenerator
  • UnresolvedGenerator
  • UserDefinedGenerator

Data Type

Expression 
dataType: DataType

dataType is part of the Expression abstraction.

dataType is an ArrayType of the elementSchema.

supportCodegen

supportCodegen: Boolean

supportCodegen is enabled (true) when this Generator is not CodegenFallback

supportCodegen is used when: