Operation¶

Operation is an abstraction of operations that can be executed on Delta tables.

Operation is described by a name and parameters (that are simply used to create a CommitInfo for OptimisticTransactionImpl when committed and, as a way to bypass a transaction, ConvertToDeltaCommand).

Operation may have performance metrics.

Contract¶

Parameters¶

parameters: Map[String, Any]

Parameters of the operation (to create a CommitInfo with the JSON-encoded values)

Used when Operation is requested for parameters with the values in JSON format

Implementations¶

AddColumns¶

AddConstraint¶

ChangeColumn¶

Convert¶

CreateTable¶

Delete¶

DropConstraint¶

ManualUpdate¶

Merge¶

ReplaceColumns¶

ReplaceTable¶

SetTableProperties¶

Name: SET TBLPROPERTIES

Parameters:

• properties

Used when:

• AlterTableSetPropertiesDeltaCommand is executed

StreamingUpdate¶

Name: STREAMING UPDATE

Parameters:

• outputMode
• queryId
• epochId

Used when:

• DeltaSink is requested to addBatch

Truncate¶

UnsetTableProperties¶

Update¶

UpdateColumnMetadata¶

UpdateSchema¶

UpgradeProtocol¶

Write¶

Merge¶

Recorded when a merge operation is committed to a Delta table (when MergeIntoCommand is executed)

Creating Instance¶

Operation takes the following to be created:

• Name of this operation

Serializing Parameter Values (to JSON Format)¶

jsonEncodedValues: Map[String, String]

jsonEncodedValues converts the values of the parameters to JSON format.

jsonEncodedValues is used when:

• OptimisticTransactionImpl is requested to commit
• ConvertToDeltaCommand command is requested to streamWrite

operationMetrics Registry¶

operationMetrics: Set[String]

operationMetrics is empty by default (and is expected to be overriden by concrete operations)

operationMetrics is used when Operation is requested to transformMetrics.

Transforming Performance Metrics¶

transformMetrics(
  metrics: Map[String, SQLMetric]): Map[String, String]

transformMetrics returns a collection of SQLMetrics (Spark SQL) and their values (as text) that are defined as the operationMetrics.

transformMetrics is used when:

• SQLMetricsReporting is requested for operation metrics