Skip to content

DeleteCommandMetrics

DeleteCommandMetrics is a marker extension of the LeafRunnableCommand (Spark SQL) abstraction for delete commands with performance metrics.

Performance Metrics

createMetrics: Map[String, SQLMetric]

createMetrics is used when:

Name web UI
numRemovedFiles number of files removed.
numAddedFiles number of files added.
numDeletedRows number of rows deleted.
numFilesBeforeSkipping number of files before skipping
numBytesBeforeSkipping number of bytes before skipping
numFilesAfterSkipping number of files after skipping
numBytesAfterSkipping number of bytes after skipping
numPartitionsAfterSkipping number of partitions after skipping
numPartitionsAddedTo number of partitions added
numPartitionsRemovedFrom number of partitions removed
numCopiedRows number of rows copied
numBytesAdded number of bytes added
numBytesRemoved number of bytes removed
executionTimeMs time taken to execute the entire operation
scanTimeMs time taken to scan the files for matches
rewriteTimeMs time taken to rewrite the matched files
numAddedChangeFiles number of change data capture files generated
changeFileBytes total size of change data capture files generated
numTouchedRows number of rows touched

getDeletedRowsFromAddFilesAndUpdateMetrics

getDeletedRowsFromAddFilesAndUpdateMetrics(
  files: Seq[AddFile]): Option[Long]

getDeletedRowsFromAddFilesAndUpdateMetrics...FIXME

getDeletedRowsFromAddFilesAndUpdateMetrics is used when: