Operation¶

Operation is an abstraction of operations that can be executed on a Delta table.

Operation is described by a name and parameters (that are simply used to create a CommitInfo for OptimisticTransactionImpl when committed and, as a way to bypass a transaction, ConvertToDeltaCommand).

Operation may have performance metrics.

Contract¶

Parameters¶

parameters: Map[String, Any]

Parameters of the operation (to create a CommitInfo with the JSON-encoded values)

Used when:

Operation is requested for the parameters with the values in JSON format

Implementations¶

Sealed Abstract Class

Operation is a Scala sealed abstract class which means that all of the implementations are in the same compilation unit (a single file).

AddColumns¶

AddConstraint¶

ChangeColumn¶

Convert¶

CreateTable¶

Delete¶

DropConstraint¶

ManualUpdate¶

Merge¶

Name: MERGE

Parameters:

predicate
matchedPredicates
notMatchedPredicates

changesData: true

Used when:

MergeIntoCommand is executed (and committed to a Delta table)

ReplaceColumns¶

ReplaceTable¶

SetTableProperties¶

Name: SET TBLPROPERTIES

Parameters:

properties

Used when:

AlterTableSetPropertiesDeltaCommand is executed

StreamingUpdate¶

Name: STREAMING UPDATE

Parameters:

outputMode
queryId
epochId

Used when:

DeltaSink is requested to addBatch

Truncate¶

UnsetTableProperties¶

Update¶

UpdateColumnMetadata¶

UpdateSchema¶

UpgradeProtocol¶

Write¶

Creating Instance¶

Operation takes the following to be created:

Name of this operation

Abstract Class

Operation is an abstract class and cannot be created directly. It is created indirectly for the concrete Operations.

Serializing Parameter Values (to JSON Format)¶

jsonEncodedValues: Map[String, String]

jsonEncodedValues converts the values of the parameters to JSON format.

jsonEncodedValues is used when:

OptimisticTransactionImpl is requested to commit
ConvertToDeltaCommand command is requested to streamWrite

Operation Metrics¶

operationMetrics: Set[String]

operationMetrics is empty by default (and is expected to be overriden by concrete operations).

operationMetrics is used when:

Operation is requested to transformMetrics

Transforming Performance Metrics¶

transformMetrics(
  metrics: Map[String, SQLMetric]): Map[String, String]

transformMetrics returns a collection of SQLMetrics (Spark SQL) and their values (as text) that are defined as the operation metrics.

transformMetrics is used when:

SQLMetricsReporting is requested for operation metrics

User Metadata¶

userMetadata: Option[String]

userMetadata is undefined (None) by default (and is expected to be overriden by concrete operations).

userMetadata is used when:

OptimisticTransactionImpl is requested for the user metadata

changesData Flag¶

changesData: Boolean

changesData is disabled (false) by default (and is expected to be overriden by concrete operations).

Note

changesData seems not used.