Skip to content

PySpark SQL

PySpark SQL is a Python module to work with Spark SQL.

from pyspark.sql import *

The above * import imports the following classes:

  • SparkSession
  • SQLContext
  • HiveContext
  • UDFRegistration
  • DataFrame
  • GroupedData
  • Column
  • Catalog
  • Observation
  • Row
  • DataFrameNaFunctions
  • DataFrameStatFunctions
  • Window
  • WindowSpec
  • DataFrameReader
  • DataFrameWriter
  • DataFrameWriterV2
  • PandasCogroupedOps