ParquetReadSupport¶

ParquetReadSupport is a ReadSupport (Apache Parquet) of UnsafeRows for non-Vectorized Parquet Decoding.

ParquetReadSupport is the value of parquet.read.support.class Hadoop configuration property for the following:

• ParquetFileFormat
• ParquetScan

Creating Instance¶

ParquetReadSupport takes the following to be created:

• ZoneId (optional)
• enableVectorizedReader
• DateTime RebaseSpec
• int96 RebaseSpec

ParquetReadSupport is created when:

• ParquetFileFormat is requested to buildReaderWithPartitionValues (with enableVectorizedReader disabled)
• ParquetPartitionReaderFactory is requested to createRowBaseParquetReader

Logging¶

Enable ALL logging level for org.apache.spark.sql.execution.datasources.parquet.ParquetReadSupport logger to see what happens inside.

Add the following line to conf/log4j2.properties:

logger.ParquetReadSupport.name = org.apache.spark.sql.execution.datasources.parquet.ParquetReadSupport
logger.ParquetReadSupport.level = all

Refer to Logging.