MergeIntoCommand

MergeIntoCommand is a Delta command that can…​FIXME

MergeIntoCommand is a logical command (RunnableCommand).

Read up on RunnableCommand in The Internals of Spark SQL online book.

MergeIntoCommand is created exclusively when DeltaMergeBuilder is requested to execute (using PreprocessTableMerge).

Creating MergeIntoCommand Instance

MergeIntoCommand takes the following to be created:

  • Source Data (LogicalPlan)

  • Target Data (LogicalPlan)

  • TahoeFileIndex

  • Condition (Expression)

  • Matched Clauses (Seq[MergeIntoMatchedClause])

  • Non-Matched Clauses (Option[MergeIntoInsertClause])

MergeIntoCommand initializes the internal properties.

Running Command — run Method

run(spark: SparkSession): Seq[Row]
run is part of the RunnableCommand contract to…​FIXME.

run…​FIXME

writeAllChanges Internal Method

writeAllChanges(
  spark: SparkSession,
  deltaTxn: OptimisticTransaction,
  filesToRewrite: Seq[AddFile]): Seq[AddFile]

writeAllChanges…​FIXME

writeAllChanges is used exclusively when MergeIntoCommand is requested to run.

findTouchedFiles Internal Method

findTouchedFiles(
  deltaTxn: OptimisticTransaction,
  files: Seq[AddFile]): LogicalPlan

findTouchedFiles…​FIXME

findTouchedFiles is used when MergeIntoCommand is requested to run.

buildTargetPlanWithFiles Internal Method

buildTargetPlanWithFiles(
  deltaTxn: OptimisticTransaction,
  files: Seq[AddFile]): LogicalPlan

buildTargetPlanWithFiles…​FIXME

buildTargetPlanWithFiles is used when MergeIntoCommand is requested to run (via findTouchedFiles and writeAllChanges).