PartitionUtils¶

createDeltaParquetReaderFactory¶

PartitionReaderFactory createDeltaParquetReaderFactory(
  Snapshot snapshot,
  StructType dataSchema,
  StructType partitionSchema,
  StructType readDataSchema,
  Filter[] dataFilters,
  scala.collection.immutable.Map<String, String> scalaOptions,
  Configuration hadoopConf,
  SQLConf sqlConf)

createDeltaParquetReaderFactory...FIXME

createDeltaParquetReaderFactory is used when:

SparkBatch is requested to create a PartitionReaderFactory
SparkMicroBatchStream is requested to create a PartitionReaderFactory

createDeltaParquetFileFormat¶

DeltaParquetFileFormatV2 createDeltaParquetFileFormat(
  Snapshot snapshot,
  String tablePath,
  boolean optimizationsEnabled,
  Option<Boolean> useMetadataRowIndex)

createDeltaParquetFileFormat creates a DeltaParquetFileFormatV2 for the given Snapshot (that is expected to be a SnapshotImpl of Delta Kernel) and the following:

DeltaParquetFileFormatV2	Value
nullableRowTrackingConstantFields	`false`
nullableRowTrackingGeneratedFields	`false`
optimizationsEnabled	Given `optimizationsEnabled`
tablePath	Given `tablePath`
isCDCRead	`false`
useMetadataRowIndex	Given `useMetadataRowIndex`