DeltaMergeIntoClause — WHEN Clause¶

DeltaMergeIntoClause is an extension of the Expression (Spark SQL) abstraction for WHEN clauses.

Contract¶

Actions¶

actions: Seq[Expression]

Action Expressions (Spark SQL)

actions can only be the following expressions:

UnresolvedStar
DeltaMergeAction

actions is either empty or given when DeltaMergeIntoClauses are created.

actions is empty for the following DeltaMergeIntoClauses:

actions is given when the following DeltaMergeIntoClauses are created:

Clause Type¶

clauseType: String

String representation of the clause type

DeltaMergeIntoClause	clauseType
DeltaMergeIntoMatchedUpdateClause	`Update`
DeltaMergeIntoMatchedDeleteClause	`Delete`
DeltaMergeIntoNotMatchedInsertClause	`Insert`
DeltaMergeIntoNotMatchedBySourceUpdateClause	`Update`
DeltaMergeIntoNotMatchedBySourceDeleteClause	`Delete`

Used when:

DeltaMergeIntoClause is requested for the string representation
MergePredicate is created
PreprocessTableMerge is executed
MergeOutputGeneration is requested to generatePrecomputedConditionsAndDF
MergeClauseStats is created

Condition¶

condition: Option[Expression]

Note

condition is always given when DeltaMergeIntoClauses are created.

Implementations¶

Sealed Trait

DeltaMergeIntoClause is a Scala sealed trait which means that all of the implementations are in the same compilation unit (a single file).

Verifing Actions¶

verifyActions(): Unit

verifyActions goes over the actions and makes sure that they are either UnresolvedStars (Spark SQL) or DeltaMergeActions.

For unsupported actions, verifyActions throws an IllegalArgumentException:

Unexpected action expression [action] in [this]

verifyActions is used when:

DeltaMergeInto is created

toActions¶

toActions(
  assignments: Seq[Assignment]): Seq[Expression]
toActions(
  colNames: Seq[UnresolvedAttribute],
  exprs: Seq[Expression],
  isEmptySeqEqualToStar: Boolean = true): Seq[Expression]

toActions requires that the numbers of colNames and exprs are the same.

toActions creates one of the following expressions based on the given colNames and isEmptySeqEqualToStar flag:

UnresolvedStar for no colNames and isEmptySeqEqualToStar flag enabled
DeltaMergeActions (for every pair of column name and expression based on colNames and exprs, respectively)

toActions is used when:

DeltaAnalysis logical resolution rule is executed
DeltaMergeMatchedActionBuilder is requested to updateAll, addUpdateClause, insertAll and addInsertClause
DeltaMergeIntoMatchedUpdateClause is created

resolvedActions¶

resolvedActions: Seq[DeltaMergeAction]

resolvedActions...FIXME

resolvedActions is used when:

PreprocessTableMerge logical resolution rule is executed
MergeIntoCommand is executed (and requested to writeInsertsOnlyWhenNoMatchedClauses and writeAllChanges)