Skip to content


SimplePythonFunction is a PythonFunction.

Creating Instance

SimplePythonFunction takes the following to be created:

SimplePythonFunction is created when:

  • SparkConnectPlanner is requested to transformPythonFunction
  • pyspark.rdd (Python module) is requested to _wrap_function
  • pyspark.sql.udf (Python module) is requested to _wrap_function

Python Executable

SimplePythonFunction is given the Python Executable when created.

The Python Executable is controlled by PYSPARK_PYTHON environment variable (in PySpark) or PYSPARK_DRIVER_PYTHON (in PySpark Connect).