Skip to content

ArrowPythonRunner

ArrowPythonRunner is a pyspark runner with PythonArrowOutput for...FIXME

Creating Instance

ArrowPythonRunner takes the following to be created:

  • Seq[ChainedPythonFunctions]
  • Eval Type
  • Argument Offsets (Array[Array[Int]])
  • Schema (Spark SQL)
  • TimeZone ID
  • Configuration (Map[String, String])

ArrowPythonRunner is created when AggregateInPandasExec, ArrowEvalPythonExec, FlatMapGroupsInPandasExec, MapInPandasExec, WindowInPandasExec physical operators are executed.


Last update: 2021-03-03