Operation¶

Operation is an abstraction of operations that can be executed on a Delta table.

Operation is described by a name and parameters (that are simply used to create a CommitInfo for OptimisticTransactionImpl when committed and, as a way to bypass a transaction, ConvertToDeltaCommand).

Operation may have performance metrics.

Contract¶

Parameters¶

parameters: Map[String, Any]

Parameters of the operation (to create a CommitInfo with the JSON-encoded values)

Used when:

• Operation is requested for the parameters with the values in JSON format

Implementations¶

AddColumns¶

AddConstraint¶

ChangeColumn¶

Convert¶

CreateTable¶

Delete¶

DropConstraint¶

ManualUpdate¶

Merge¶

Name: MERGE

Parameters:

• predicate
• matchedPredicates
• notMatchedPredicates

changesData: true

Used when:

• MergeIntoCommand is executed (and committed to a Delta table)

ReplaceColumns¶

ReplaceTable¶

SetTableProperties¶

Name: SET TBLPROPERTIES

Parameters:

• properties

Used when:

• AlterTableSetPropertiesDeltaCommand is executed

StreamingUpdate¶

Name: STREAMING UPDATE

Parameters:

• outputMode
• queryId
• epochId

Used when:

• DeltaSink is requested to addBatch

Truncate¶

UnsetTableProperties¶

Update¶

UpdateColumnMetadata¶

UpdateSchema¶

UpgradeProtocol¶

Write¶

Creating Instance¶

Operation takes the following to be created:

• Name of this operation

Serializing Parameter Values (to JSON Format)¶

jsonEncodedValues: Map[String, String]

jsonEncodedValues converts the values of the parameters to JSON format.

jsonEncodedValues is used when:

• OptimisticTransactionImpl is requested to commit
• ConvertToDeltaCommand command is requested to streamWrite

Operation Metrics¶

operationMetrics: Set[String]

operationMetrics is empty by default (and is expected to be overriden by concrete operations).

operationMetrics is used when:

• Operation is requested to transformMetrics

Transforming Performance Metrics¶

transformMetrics(
  metrics: Map[String, SQLMetric]): Map[String, String]

transformMetrics returns a collection of SQLMetrics (Spark SQL) and their values (as text) that are defined as the operation metrics.

transformMetrics is used when:

• SQLMetricsReporting is requested for operation metrics

User Metadata¶

userMetadata: Option[String]

userMetadata is undefined (None) by default (and is expected to be overriden by concrete operations).

userMetadata is used when:

• OptimisticTransactionImpl is requested for the user metadata

changesData Flag¶

changesData: Boolean

changesData is disabled (false) by default (and is expected to be overriden by concrete operations).