ClusteredTableUtilsBase (ClusteredTableUtils)¶

clusteringColumns¶

ClusteredTableUtilsBase defines clusteringColumns value for the name of the table property with the clustering columns of a delta table.

clusteringColumns is used when:

ClusterBySpec is requested to toProperty
ClusteredTableUtilsBase is requested to getClusterBySpecOptional, removeClusteringColumnsProperty

Clustering Provider¶

clusteringProvider: String

clusteringProvider is always liquid.

clusteringProvider is used when:

OptimizeExecutor is requested to runOptimizeBinJob

removeClusteringColumnsProperty¶

removeClusteringColumnsProperty(
  configuration: Map[String, String]): Map[String, String]

removeClusteringColumnsProperty...FIXME

removeClusteringColumnsProperty is used when:

CreateDeltaTableCommand is requested for the provided metadata

Domain Metadata¶

getDomainMetadataOptional(
  table: CatalogTable,
  txn: OptimisticTransaction): Option[DomainMetadata] // (1)!
getDomainMetadataOptional(
  clusterBySpecOpt: Option[ClusterBySpec],
  txn: OptimisticTransaction): Option[DomainMetadata]

Uses getClusterBySpecOptional with the given CatalogTable (Spark SQL)

getDomainMetadataOptional validateClusteringColumnsInStatsSchema in the given ClusterBySpec, if specified.

getDomainMetadataOptional createDomainMetadata with the column names (in the given ClusterBySpec).

getDomainMetadataOptional is used when:

CreateDeltaTableCommand is requested to handleCreateTable, handleCreateTableAsSelect

Creating DomainMetadata¶

createDomainMetadata(
  clusteringColumns: Seq[ClusteringColumn]): DomainMetadata

createDomainMetadata creates a ClusteringMetadataDomain from the given ClusteringColumns and converts it to a DomainMetadata.

getClusterBySpecOptional¶

getClusterBySpecOptional(
  table: CatalogTable): Option[ClusterBySpec]

getClusterBySpecOptional...FIXME

getClusterBySpecOptional is used when:

CreateDeltaTableCommand is requested to handleCreateTableAsSelect
ClusteredTableUtilsBase is requested for the domain metadata

getClusteringColumnsAsProperty¶

getClusteringColumnsAsProperty(
  maybeClusterBySpec: Option[ClusterBySpec]): Option[(String, String)]

getClusteringColumnsAsProperty...FIXME

getClusteringColumnsAsProperty is used when:

DeltaCatalog is requested to verifyTableAndSolidify

getTableFeatureProperties¶

getTableFeatureProperties(
  existingProperties: Map[String, String]): Map[String, String]

getTableFeatureProperties...FIXME

getTableFeatureProperties is used when:

DeltaCatalog is requested to verifyTableAndSolidify

isSupported¶

isSupported(
  protocol: Protocol): Boolean

isSupported requests the given Protocol to isFeatureSupported with ClusteringTableFeature.

isSupported is used when:

CreateDeltaTableCommand is requested to validatePrerequisitesForClusteredTable
OptimizeExecutor is requested to isClusteredTable
Optimize command is executed
ClusteredTableUtilsBase is requested to validatePreviewEnabled

validatePreviewEnabled¶

validatePreviewEnabled(
  protocol: Protocol): Unit
validatePreviewEnabled(
  maybeClusterBySpec: Option[ClusterBySpec]): Unit

Procedure

validatePreviewEnabled is a procedure (returns Unit) so what happens inside stays inside (paraphrasing the former advertising slogan of Las Vegas, Nevada).

validatePreviewEnabled...FIXME

validatePreviewEnabled is used when:

Optimize command is executed
WriteIntoDelta command is requested to write data out (to a delta table)
DeltaCatalog is requested to create a delta table (and validateClusterBySpec)