DeltaTableOperations — Delta DML Operations¶

DeltaTableOperations is an abstraction of management services for executing delete, generate, history, update, and vacuum operations (commands).

DeltaTableOperations is assumed to be associated with a DeltaTable.

Implementations¶

• DeltaTable

Executing DeleteFromTable Command¶

executeDelete(
  condition: Option[Expression]): Unit

executeDelete creates a DataFrame for a DeleteFromTable logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition).

executeDelete is used for DeltaTable.delete operator.

Executing DeltaGenerateCommand Command¶

executeGenerate(
  tblIdentifier: String,
  mode: String): Unit

executeGenerate requests the SQL parser (of the SparkSession) to parse the given table identifier, creates a DeltaGenerateCommand and runs it.

executeGenerate is used for DeltaTable.generate operator.

Executing History Command¶

executeHistory(
  deltaLog: DeltaLog,
  limit: Option[Int]): DataFrame

executeHistory creates DeltaHistoryManager (for the given DeltaLog) and requests it for the number of commits to match the limit. In the end, executeHistory creates a DataFrame for the commits.

executeHistory is used for DeltaTable.history operator.

Executing UpdateTable Command¶

executeUpdate(
  set: Map[String, Column],
  condition: Option[Column]): Unit

executeUpdate creates a DataFrame for a UpdateTable logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition).

executeUpdate is used for DeltaTable.update and DeltaTable.updateExpr operators.

Executing VacuumCommand¶

executeVacuum(
  deltaLog: DeltaLog,
  retentionHours: Option[Double]): DataFrame

executeVacuum uses the VacuumCommand utility to gc (with the dryRun flag off and the given retentionHours).

In the end, executeVacuum returns an empty DataFrame.

NOTE: executeVacuum is used exclusively in <> operator.