UpdateCommand¶
UpdateCommand
is a DeltaCommand that represents DeltaUpdateTable logical command at execution.
UpdateCommand
is a RunnableCommand
(Spark SQL) logical operator.
Creating Instance¶
UpdateCommand
takes the following to be created:
- TahoeFileIndex
- Target Data (LogicalPlan)
- Update Expressions (Spark SQL)
- (optional) Condition Expression (Spark SQL)
UpdateCommand
is created when:
- PreprocessTableUpdate logical resolution rule is executed (and resolves a DeltaUpdateTable logical command)
Performance Metrics¶
Name | web UI |
---|---|
numAddedFiles | number of files added. |
numRemovedFiles | number of files removed. |
numUpdatedRows | number of rows updated. |
executionTimeMs | time taken to execute the entire operation |
scanTimeMs | time taken to scan the files for matches |
rewriteTimeMs | time taken to rewrite the matched files |
Executing Command¶
run(
sparkSession: SparkSession): Seq[Row]
run
is part of the RunnableCommand
(Spark SQL) abstraction.
run
...FIXME
performUpdate¶
performUpdate(
sparkSession: SparkSession,
deltaLog: DeltaLog,
txn: OptimisticTransaction): Unit
performUpdate
...FIXME
rewriteFiles¶
rewriteFiles(
spark: SparkSession,
txn: OptimisticTransaction,
rootPath: Path,
inputLeafFiles: Seq[String],
nameToAddFileMap: Map[String, AddFile],
condition: Expression): Seq[FileAction]
rewriteFiles
...FIXME
buildUpdatedColumns¶
buildUpdatedColumns(
condition: Expression): Seq[Column]
buildUpdatedColumns
...FIXME