DeltaTableOperations — Delta DML Operations¶
DeltaTableOperations
is an abstraction of management services for executing delete, generate, history, update, and vacuum operations (commands).
DeltaTableOperations
is assumed to be associated with a DeltaTable.
Implementations¶
Executing DeleteFromTable Command¶
executeDelete(
condition: Option[Expression]): Unit
executeDelete
creates a DataFrame
for a DeleteFromTable
logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition
).
Note
DeleteFromTable
is a Command
(Spark SQL) that represents DELETE FROM
SQL statement for v2 tables. As a Command
it is executed eagerly.
executeDelete
is used for DeltaTable.delete operator.
Executing DeltaGenerateCommand Command¶
executeGenerate(
tblIdentifier: String,
mode: String): Unit
executeGenerate
requests the SQL parser (of the SparkSession
) to parse the given table identifier, creates a DeltaGenerateCommand and runs it.
executeGenerate
is used for DeltaTable.generate operator.
Executing History Command¶
executeHistory(
deltaLog: DeltaLog,
limit: Option[Int]): DataFrame
executeHistory
creates DeltaHistoryManager (for the given DeltaLog) and requests it for the number of commits to match the limit
. In the end, executeHistory
creates a DataFrame
for the commits.
executeHistory
is used for DeltaTable.history operator.
Executing UpdateTable Command¶
executeUpdate(
set: Map[String, Column],
condition: Option[Column]): Unit
executeUpdate
creates a DataFrame
for a UpdateTable
logical operator (from Spark SQL) with (the analyzed logical plan of the DeltaTable and the given condition
).
Note
UpdateTable
is a Command
(Spark SQL) that represents UPDATE TABLE
SQL statement for v2 tables. As a Command
it is executed eagerly.
executeUpdate
is used for DeltaTable.update and DeltaTable.updateExpr operators.
Executing VacuumCommand¶
executeVacuum(
deltaLog: DeltaLog,
retentionHours: Option[Double]): DataFrame
executeVacuum
uses the VacuumCommand
utility to gc (with the dryRun
flag off and the given retentionHours
).
In the end, executeVacuum
returns an empty DataFrame
.
Note
executeVacuum
returns an empty DataFrame
not the one from VacuumCommand.gc.
NOTE: executeVacuum
is used exclusively in <