Skip to content

DeltaTableOperations — Delta DML Operations

DeltaTableOperations is an abstraction of management services for executing delete, generate, history, update, and vacuum operations (commands).

DeltaTableOperations is assumed to be associated with a DeltaTable.


Executing DeleteFromTable Command

  condition: Option[Expression]): Unit

executeDelete creates a DataFrame for a DeleteFromTable logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition).


DeleteFromTable is a Command (Spark SQL) that represents DELETE FROM SQL statement for v2 tables. As a Command it is executed eagerly.

executeDelete is used for DeltaTable.delete operator.

Executing DeltaGenerateCommand Command

  tblIdentifier: String,
  mode: String): Unit

executeGenerate requests the SQL parser (of the SparkSession) to parse the given table identifier, creates a DeltaGenerateCommand and runs it.

executeGenerate is used for DeltaTable.generate operator.

Executing History Command

  deltaLog: DeltaLog,
  limit: Option[Int]): DataFrame

executeHistory creates DeltaHistoryManager (for the given DeltaLog) and requests it for the number of commits to match the limit. In the end, executeHistory creates a DataFrame for the commits.

executeHistory is used for DeltaTable.history operator.

Executing UpdateTable Command

  set: Map[String, Column],
  condition: Option[Column]): Unit

executeUpdate creates a DataFrame for an UpdateTable (Spark SQL) logical operator with the analyzed logical plan of the DeltaTable and the given condition.


UpdateTable represents UPDATE TABLE SQL statement for v2 tables and is executed eagerly.

executeUpdate is used for DeltaTable.update and DeltaTable.updateExpr operators.

Executing VacuumCommand

  deltaLog: DeltaLog,
  retentionHours: Option[Double]): DataFrame

executeVacuum uses the VacuumCommand utility to gc (with the dryRun flag off and the given retentionHours).

In the end, executeVacuum returns an empty DataFrame (not the one from VacuumCommand.gc).

executeVacuum is used for DeltaTable.vacuum operator.

Last update: 2021-03-16