Skip to content

DeltaTableIdentifier

DeltaTableIdentifier is an identifier of a delta table by TableIdentifier or directory depending whether it is a catalog table or not (and living non-cataloged).

Creating Instance

DeltaTableIdentifier takes the following to be created:

  • Path to a delta table (default: undefined)
  • TableIdentifier (default: undefined)

Either a path or a table identifier is required.

Creating DeltaTableIdentifier

apply(
  spark: SparkSession,
  identifier: TableIdentifier): Option[DeltaTableIdentifier]

apply creates a new DeltaTableIdentifier for the given TableIdentifier:

  1. For a path (to a delta table), apply creates a DeltaTableIdentifier with the path
  2. For a delta table, apply creates a DeltaTableIdentifier with a TableIdentifier
  3. For all the other cases, apply returns None

isDeltaPath

isDeltaPath(
  spark: SparkSession,
  identifier: TableIdentifier): Boolean

isDeltaPath checks whether the input TableIdentifier represents an (absolute) path to a delta table.

isDeltaPath is positive (true) when all the following hold:

  1. spark.sql.runSQLOnFiles (Spark SQL) configuration property is true
  2. DeltaSourceUtils.isDeltaTable(identifier.database)
  3. The TableIdentifier is not a temporary view
  4. The table in the database (as specified in the TableIdentifier) does not exist
  5. The table part (of the TableIdentifier) is absolute (starts with /)

Creating DeltaLog

getDeltaLog(
  spark: SparkSession): DeltaLog

getDeltaLog creates a DeltaLog (for the location).

Note

getDeltaLog does not seem to be used.

Location Path

getPath(
  spark: SparkSession): Path

getPath creates a Hadoop Path for the path if defined or requests SessionCatalog (Spark SQL) for the table metadata and uses the locationUri.