DeltaTableIdentifier¶

DeltaTableIdentifier is an identifier of a delta table by TableIdentifier or directory depending whether it is a catalog table or not (and living non-cataloged).

Creating Instance¶

DeltaTableIdentifier takes the following to be created:

• Path to a delta table (default: undefined)
• TableIdentifier (default: undefined)

Either a path or a table identifier is required.

Creating DeltaTableIdentifier¶

apply(
  spark: SparkSession,
  identifier: TableIdentifier): Option[DeltaTableIdentifier]

apply creates a new DeltaTableIdentifier for the given TableIdentifier:

1. For a path (to a delta table), apply creates a DeltaTableIdentifier with the path
2. For a delta table, apply creates a DeltaTableIdentifier with a TableIdentifier
3. For all the other cases, apply returns None

isDeltaPath¶

isDeltaPath(
  spark: SparkSession,
  identifier: TableIdentifier): Boolean

isDeltaPath checks whether the input TableIdentifier represents an (absolute) path to a delta table.

isDeltaPath is positive (true) when all the following hold:

1. spark.sql.runSQLOnFiles (Spark SQL) configuration property is true
2. DeltaSourceUtils.isDeltaTable(identifier.database)
3. The TableIdentifier is not a temporary view
4. The table in the database (as specified in the TableIdentifier) does not exist
5. The table part (of the TableIdentifier) is absolute (starts with /)

Creating DeltaLog¶

getDeltaLog(
  spark: SparkSession): DeltaLog

getDeltaLog creates a DeltaLog (for the location).

Location Path¶

getPath(
  spark: SparkSession): Path

getPath creates a Hadoop Path for the path if defined or requests SessionCatalog (Spark SQL) for the table metadata and uses the locationUri.