DeltaTableUtils Utility

DeltaTableUtils comes with the following utilities:

Checking Out Whether Directory Is Part of Delta Table — isDeltaTable Utility

isDeltaTable(
  spark: SparkSession,
  path: Path): Boolean

isDeltaTable tries to find the root directory of the delta table for the given path and returns whether it was successful or not.

isDeltaTable is used when DeltaTable utility is used to create a DeltaTable or check whether a directory is part of delta table or not.

Finding Root Directory Of Delta Table — findDeltaTableRoot Utility

findDeltaTableRoot(
  spark: SparkSession,
  path: Path): Option[Path]

findDeltaTableRoot traverses the Hadoop DFS-compliant path upwards (to the root directory of the file system) until _delta_log or _samples directories are found, or the root directory is reached.

For _delta_log or _samples directories, findDeltaTableRoot returns the parent directory.

findDeltaTableRoot is used when:

resolveTimeTravelVersion Utility

resolveTimeTravelVersion(
  conf: SQLConf,
  deltaLog: DeltaLog,
  tt: DeltaTimeTravelSpec): (Long, String)

resolveTimeTravelVersion…​FIXME

resolveTimeTravelVersion is used exclusively when DeltaLog is requested to create a relation (per partition filters and time travel).

splitMetadataAndDataPredicates Utility

splitMetadataAndDataPredicates(
  condition: Expression,
  partitionColumns: Seq[String],
  spark: SparkSession): (Seq[Expression], Seq[Expression])

splitMetadataAndDataPredicates…​FIXME

splitMetadataAndDataPredicates is used when:

isPredicatePartitionColumnsOnly Utility

isPredicatePartitionColumnsOnly(
  condition: Expression,
  partitionColumns: Seq[String],
  spark: SparkSession): Boolean

isPredicatePartitionColumnsOnly…​FIXME

isPredicatePartitionColumnsOnly is used when…​FIXME

combineWithCatalogMetadata Utility

combineWithCatalogMetadata(
  sparkSession: SparkSession,
  table: CatalogTable): CatalogTable

combineWithCatalogMetadata…​FIXME

combineWithCatalogMetadata seems unused.