Skip to content

DeltaTableOperations — Delta DML Operations

DeltaTableOperations is an abstraction of management services for executing delete, generate, history, update, and vacuum operations (commands).

DeltaTableOperations is assumed to be associated with a DeltaTable.

Implementations

Executing DeleteFromTable Command

executeDelete(
  condition: Option[Expression]): Unit

executeDelete creates a DataFrame for a DeleteFromTable logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition).

Note

DeleteFromTable is a Command (Spark SQL) that represents DELETE FROM SQL statement for v2 tables. As a Command it is executed eagerly.

executeDelete is used for DeltaTable.delete operator.

Executing DeltaGenerateCommand Command

executeGenerate(
  tblIdentifier: String,
  mode: String): Unit

executeGenerate requests the SQL parser (of the SparkSession) to parse the given table identifier, creates a DeltaGenerateCommand and runs it.

executeGenerate is used for DeltaTable.generate operator.

Executing History Command

executeHistory(
  deltaLog: DeltaLog,
  limit: Option[Int]): DataFrame

executeHistory creates DeltaHistoryManager (for the given DeltaLog) and requests it for the number of commits to match the limit. In the end, executeHistory creates a DataFrame for the commits.

executeHistory is used for DeltaTable.history operator.

Executing UpdateTable Command

executeUpdate(
  set: Map[String, Column],
  condition: Option[Column]): Unit

executeUpdate creates a DataFrame for a UpdateTable logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition).

Note

UpdateTable is a Command (Spark SQL) that represents UPDATE TABLE SQL statement for v2 tables. As a Command it is executed eagerly.

executeUpdate is used for DeltaTable.update and DeltaTable.updateExpr operators.

Executing VacuumCommand

executeVacuum(
  deltaLog: DeltaLog,
  retentionHours: Option[Double]): DataFrame

executeVacuum uses the VacuumCommand utility to gc (with the dryRun flag off and the given retentionHours).

In the end, executeVacuum returns an empty DataFrame.

Note

executeVacuum returns an empty DataFrame not the one from VacuumCommand.gc.

NOTE: executeVacuum is used exclusively in <> operator.


Last update: 2020-09-30