Skip to content

UpdateCommand

UpdateCommand is a DeltaCommand that represents DeltaUpdateTable logical command at execution.

UpdateCommand is a RunnableCommand (Spark SQL) logical operator.

Creating Instance

UpdateCommand takes the following to be created:

UpdateCommand is created when:

Performance Metrics

Name web UI
numAddedFiles number of files added.
numRemovedFiles number of files removed.
numUpdatedRows number of rows updated.
executionTimeMs time taken to execute the entire operation
scanTimeMs time taken to scan the files for matches
rewriteTimeMs time taken to rewrite the matched files

Executing Command

run(
  sparkSession: SparkSession): Seq[Row]

run is part of the RunnableCommand (Spark SQL) abstraction.

run...FIXME

performUpdate

performUpdate(
  sparkSession: SparkSession,
  deltaLog: DeltaLog,
  txn: OptimisticTransaction): Unit

performUpdate...FIXME

rewriteFiles

rewriteFiles(
  spark: SparkSession,
  txn: OptimisticTransaction,
  rootPath: Path,
  inputLeafFiles: Seq[String],
  nameToAddFileMap: Map[String, AddFile],
  condition: Expression): Seq[FileAction]

rewriteFiles...FIXME

buildUpdatedColumns

buildUpdatedColumns(
  condition: Expression): Seq[Column]

buildUpdatedColumns...FIXME


Last update: 2021-05-21
Back to top