Skip to content

Operation

Operation is an abstraction of operations that can be executed on Delta tables.

Operation is described by a name and parameters (that are simply used to create a CommitInfo for OptimisticTransactionImpl when committed and, as a way to bypass a transaction, ConvertToDeltaCommand).

Operation may have performance metrics.

Contract

parameters

parameters: Map[String, Any]

Parameters of the operation (to create a CommitInfo with the JSON-encoded values)

Used when Operation is requested for parameters with the values in JSON format

Implementations

Sealed Abstract Class

Operation is a Scala sealed abstract class which means that all of the implementations are in the same compilation unit (a single file).

  • AddColumns
  • ChangeColumn
  • ComputeStats
  • Convert
  • CreateTable
  • Delete
  • FileNotificationRetention
  • Fsck
  • ManualUpdate
  • Optimize
  • ReplaceColumns
  • ReplaceTable
  • ResetZCubeInfo
  • SetTableProperties
  • StreamingUpdate
  • Truncate
  • UnsetTableProperties
  • Update
  • UpdateColumnMetadata
  • UpdateSchema
  • UpgradeProtocol
  • Write

Merge

Recorded when a merge operation is committed to a Delta table (when MergeIntoCommand is executed)

Creating Instance

Operation takes the following to be created:

  • Name
Abstract Class

Operation is an abstract class and cannot be created directly. It is created indirectly for the concrete Operations.

Serializing Parameter Values (to JSON Format)

jsonEncodedValues: Map[String, String]

jsonEncodedValues converts the values of the parameters to JSON format.

jsonEncodedValues is used when:

  • OptimisticTransactionImpl is requested to commit
  • ConvertToDeltaCommand command is requested to streamWrite

operationMetrics Registry

operationMetrics: Set[String]

operationMetrics is empty by default (and is expected to be overriden by concrete operations)

operationMetrics is used when Operation is requested to transformMetrics.

transformMetrics Method

transformMetrics(
  metrics: Map[String, SQLMetric]): Map[String, String]

transformMetrics returns a collection of performance metrics (SQLMetric) and their values (as a text) that are defined as the operationMetrics.

transformMetrics is used when SQLMetricsReporting is requested to getMetricsForOperation.


Last update: 2020-09-24