Skip to content

DeltaColumnMappingBase (DeltaColumnMapping)

DeltaColumnMappingBase is an abstraction of DeltaColumnMappings.

Implementations

createPhysicalSchema

createPhysicalSchema(
  schema: StructType,
  referenceSchema: StructType,
  columnMappingMode: DeltaColumnMappingMode,
  checkSupportedMode: Boolean = true): StructType

createPhysicalSchema...FIXME

createPhysicalSchema is used when:

renameColumns

renameColumns(
  schema: StructType): StructType

renameColumns...FIXME

renameColumns is used when:

requiresNewProtocol

requiresNewProtocol(
  metadata: Metadata): Boolean

requiresNewProtocol...FIXME

requiresNewProtocol is used when:

checkColumnIdAndPhysicalNameAssignments

checkColumnIdAndPhysicalNameAssignments(
  schema: StructType,
  mode: DeltaColumnMappingMode): Unit

checkColumnIdAndPhysicalNameAssignments...FIXME

checkColumnIdAndPhysicalNameAssignments is used when:

dropColumnMappingMetadata

dropColumnMappingMetadata(
  schema: StructType): StructType

dropColumnMappingMetadata...FIXME

dropColumnMappingMetadata is used when:

Mapping Virtual to Physical Field Name

getPhysicalName(
  field: StructField): String

getPhysicalName requests the given StructField (Spark SQL) for the Metadata to extract delta.columnMapping.physicalName key, if available (for column mapping). Otherwise, getPhysicalName returns the name of the given StructField (with no name changes).

getPhysicalName is used when:

verifyAndUpdateMetadataChange

verifyAndUpdateMetadataChange(
  oldProtocol: Protocol,
  oldMetadata: Metadata,
  newMetadata: Metadata,
  isCreatingNewTable: Boolean): Metadata

verifyAndUpdateMetadataChange...FIXME

In the end, verifyAndUpdateMetadataChange tryFixMetadata with the given newMetadata and oldMetadata metadata.

verifyAndUpdateMetadataChange is used when:

tryFixMetadata

tryFixMetadata(
  oldMetadata: Metadata,
  newMetadata: Metadata,
  isChangingModeOnExistingTable: Boolean): Metadata

tryFixMetadata reads columnMapping.mode table property from the given newMetadata table metadata.

If the DeltaColumnMappingMode is IdMapping or NameMapping, tryFixMetadata assignColumnIdAndPhysicalName with the given newMetadata and oldMetadata metadata and isChangingModeOnExistingTable flag.

For NoMapping, tryFixMetadata does nothing and returns the given newMetadata.

DeltaColumnMapping

DeltaColumnMapping is the only DeltaColumnMappingBase.