ProtocolMetadataAdapter¶

ProtocolMetadataAdapter is an abstraction of adapters for DeltaParquetFileFormatBase to work alongside the two Spark Connector implementations:

• Spark Connector V1
• Kernel-based Spark Connector V2

ProtocolMetadataAdapter hides the implementation details of Protocol and Metadata classes between the Delta Spark connectors.

Contract¶

assertTableReadable¶

assertTableReadable(
  sparkSession: SparkSession): Unit

Used when:

• DeltaParquetFileFormatBase is created

columnMappingMode¶

columnMappingMode: DeltaColumnMappingMode

DeltaColumnMappingMode

Used when:

• DeltaParquetFileFormatBase is created

createRowTrackingMetadataFields¶

createRowTrackingMetadataFields(
  nullableRowTrackingConstantFields: Boolean,
  nullableRowTrackingGeneratedFields: Boolean): Iterable[StructField]

Metadata columns for Row Tracking

Used when:

• DeltaParquetFileFormatBase is requested for the metadataSchemaFields

getReferenceSchema¶

getReferenceSchema: StructType

Used when:

• DeltaParquetFileFormatBase is created

isDeletionVectorReadable¶

isDeletionVectorReadable: Boolean

Whether Deletion Vectors are readable on this table

Used when:

• DeltaParquetFileFormatBase is requested for the metadataSchemaFields

isIcebergCompatAnyEnabled¶

isIcebergCompatAnyEnabled: Boolean

Whether any version of IcebergCompat is enabled on this table

Used when:

• DeltaParquetFileFormatBase is requested to prepareWrite

isIcebergCompatGeqEnabled¶

isIcebergCompatGeqEnabled(
  version: Int): Boolean

Whether IcebergCompat is enabled at or above the specified version

Used when:

• DeltaParquetFileFormatBase is requested for the prepareWrite

isRowIdEnabled¶

isRowIdEnabled: Boolean

Whether Row Tracking is enabled on this table

Used when:

• DeltaParquetFileFormatBase is requested for the metadataSchemaFields

Implementations¶

• ProtocolMetadataAdapterV1
• ProtocolMetadataAdapterV2