DeltaTableOperations — Delta DML Operations¶

DeltaTableOperations is an abstraction of management services for executing delete, generate, history, update, and vacuum operations (commands).

DeltaTableOperations is assumed to be associated with a DeltaTable.

Implementations¶

• DeltaTable

Executing DeleteFromTable Command¶

executeDelete(
  condition: Option[Expression]): Unit

executeDelete creates a DataFrame for a DeleteFromTable logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition).

executeDelete is used for DeltaTable.delete operator.

Executing DeltaGenerateCommand Command¶

executeGenerate(
  tblIdentifier: String,
  mode: String): Unit

executeGenerate requests the SQL parser (of the SparkSession) to parse the given table identifier, creates a DeltaGenerateCommand and runs it.

executeGenerate is used for DeltaTable.generate operator.

Executing History Command¶

executeHistory(
  deltaLog: DeltaLog,
  limit: Option[Int]): DataFrame

executeHistory creates DeltaHistoryManager (for the given DeltaLog) and requests it for the number of commits to match the limit. In the end, executeHistory creates a DataFrame for the commits.

executeHistory is used for DeltaTable.history operator.

Executing UpdateTable Command¶

executeUpdate(
  set: Map[String, Column],
  condition: Option[Column]): Unit

executeUpdate creates a DataFrame for an UpdateTable (Spark SQL) logical operator with the analyzed logical plan of the DeltaTable and the given condition.

executeUpdate is used for DeltaTable.update and DeltaTable.updateExpr operators.

Executing VacuumCommand¶

executeVacuum(
  deltaLog: DeltaLog,
  retentionHours: Option[Double]): DataFrame

executeVacuum uses the VacuumCommand utility to gc (with the dryRun flag off and the given retentionHours).

In the end, executeVacuum returns an empty DataFrame (not the one from VacuumCommand.gc).

executeVacuum is used for DeltaTable.vacuum operator.