Skip to content

DeltaMergeBuilder

DeltaMergeBuilder is a builder interface to describe how to merge data from a source DataFrame into the target delta table (using whenMatched, whenNotMatched, and whenNotMatchedBySource conditions).

In the end, DeltaMergeBuilder is supposed to be executed to take action.

DeltaMergeBuilder creates a DeltaMergeInto logical command that is resolved to a MergeIntoCommand runnable logical command (using PreprocessTableMerge logical resolution rule).

Creating Instance

DeltaMergeBuilder takes the following to be created:

DeltaMergeBuilder is created using DeltaTable.merge operator.

Operators

whenMatched

whenMatched(): DeltaMergeMatchedActionBuilder
whenMatched(
  condition: Column): DeltaMergeMatchedActionBuilder
whenMatched(
  condition: String): DeltaMergeMatchedActionBuilder

Creates a DeltaMergeMatchedActionBuilder (for this DeltaMergeBuilder and a condition)

whenNotMatched

whenNotMatched(): DeltaMergeNotMatchedActionBuilder
whenNotMatched(
  condition: Column): DeltaMergeNotMatchedActionBuilder
whenNotMatched(
  condition: String): DeltaMergeNotMatchedActionBuilder

Creates a DeltaMergeNotMatchedActionBuilder (for this DeltaMergeBuilder and a condition)

whenNotMatchedBySource

whenNotMatchedBySource(): DeltaMergeNotMatchedBySourceActionBuilder
whenNotMatchedBySource(
  condition: Column): DeltaMergeNotMatchedBySourceActionBuilder
whenNotMatchedBySource(
  condition: String): DeltaMergeNotMatchedBySourceActionBuilder

Creates a DeltaMergeNotMatchedBySourceActionBuilder (for this DeltaMergeBuilder and a condition)

Executing Merge

execute(): Unit

execute creates a merge plan (that is DeltaMergeInto logical command) and resolves column references.

execute runs PreprocessTableMerge logical resolution rule on the DeltaMergeInto logical command (that gives MergeIntoCommand runnable logical command).

In the end, execute executes the MergeIntoCommand logical command.

Creating Logical Plan for Merge

mergePlan: DeltaMergeInto

mergePlan creates a DeltaMergeInto logical command.

mergePlan is used when DeltaMergeBuilder is requested to execute.

Creating DeltaMergeBuilder

apply(
  targetTable: DeltaTable,
  source: DataFrame,
  onCondition: Column): DeltaMergeBuilder

apply utility creates a new DeltaMergeBuilder for the given parameters and no DeltaMergeIntoClauses.

apply is used for DeltaTable.merge operator.

Adding DeltaMergeIntoClause

withClause(
  clause: DeltaMergeIntoClause): DeltaMergeBuilder

withClause creates a new DeltaMergeBuilder (based on the existing properties, e.g. the DeltaTable) with the given DeltaMergeIntoClause added to the existing DeltaMergeIntoClauses (to create a more refined DeltaMergeBuilder).

withClause is used when: