ParquetTable is a FileTable of ParquetDataSourceV2 in Parquet Data Source.
ParquetTable uses ParquetScanBuilder for scanning and ParquetWrite for writing.
ParquetTable takes the following to be created:
- Case-insensitive options
- User-specified schema
- Fallback FileFormat
ParquetTable is created when:
ParquetDataSourceV2is requested for a Table
formatName is part of the FileTable abstraction.
formatName is the following text:
inferSchema( files: Seq[FileStatus]): Option[StructType]
inferSchema is part of the FileTable abstraction.
inferSchema infers the schema (with the options and the input Hadoop
newScanBuilder( options: CaseInsensitiveStringMap): ParquetScanBuilder
newScanBuilder is part of the SupportsRead abstraction.
newScanBuilder creates a ParquetScanBuilder with the following:
newWriteBuilder( info: LogicalWriteInfo): WriteBuilder
newWriteBuilder is part of the SupportsWrite abstraction.
newWriteBuilder creates a WriteBuilder that creates a ParquetWrite (when requested to build a Write).
supportsDataType( dataType: DataType): Boolean
supportsDataType is part of the FileTable abstraction.
supportsDataType supports all AtomicTypes and the following complex DataTypes with