ArrowEvalPython Logical Operator¶

ArrowEvalPython is a BaseEvalPython unary logical operator that evaluates scalar PythonUDFs with Apache Arrow.

ArrowEvalPython is planned as ArrowEvalPythonExec physical operator.

Creating Instance¶

ArrowEvalPython takes the following to be created:

• Scalar PythonUDFs
• Result Attributes (Spark SQL)
• Child LogicalPlan (Spark SQL)
• Eval Type

ArrowEvalPython is created when:

• ExtractPythonUDFs logical optimization is executed (and requested to extract scalar PythonUDFs from a logical query plan)

evalType¶

evalType: Int

ArrowEvalPython is given an evalType when created that can only be one of the following:

• SQL_SCALAR_PANDAS_UDF
• SQL_SCALAR_PANDAS_ITER_UDF