Skip to content

ParquetScanBuilder

ParquetScanBuilder is a FileScanBuilder that SupportsPushDownFilters.

Creating Instance

ParquetScanBuilder takes the following to be created:

ParquetScanBuilder is created when:

Building Scan

build(): Scan

build creates a ParquetScan (with the readDataSchema, the readPartitionSchema and the pushedParquetFilters).

build is part of the ScanBuilder abstraction.

Pushed Filters

pushedFilters(): Array[Filter]

pushedFilters is the pushedParquetFilters.

pushedFilters is part of the SupportsPushDownFilters abstraction.

pushedParquetFilters

pushedParquetFilters: Array[Filter]
Lazy Value

pushedParquetFilters is a Scala lazy value to guarantee that the code to initialize it is executed once only (when accessed for the first time) and the computed value never changes afterwards.

Learn more in the Scala Language Specification.

pushedParquetFilters creates a ParquetFilters with the readDataSchema (converted) and the following configuration properties:

pushedParquetFilters requests the ParquetFilters for the convertibleFilters.

pushedParquetFilters is used when:

supportsNestedSchemaPruning

supportsNestedSchemaPruning: Boolean

supportsNestedSchemaPruning is true.

supportsNestedSchemaPruning is part of the FileScanBuilder abstraction.