EvalPythonExec Physical Operators¶

EvalPythonExec is an extension of the UnaryExecNode abstraction for unary physical operators that can evaluate PythonUDFs.

Contract¶

Evaluating PythonUDFs¶

evaluate(
  funcs: Seq[ChainedPythonFunctions],
  argOffsets: Array[Array[Int]],
  iter: Iterator[InternalRow],
  schema: StructType,
  context: TaskContext): Iterator[InternalRow]

Evaluates PythonUDFs (and produces internal binary rows)

Used when EvalPythonExec physical operator is executed

Implementations¶

• ArrowEvalPythonExec
• BatchEvalPythonExec

Creating Instance¶

EvalPythonExec takes the following to be created:

• PythonUDFs
• Result Attributes
• Child physical operator