AppendData Logical Command¶

AppendData is a V2WriteCommand that represents appending data (the result of executing a structured query) to a table (with the columns matching by name or position).

Creating Instance¶

AppendData takes the following to be created:

• NamedRelation for the table (to append data to)
• Query (LogicalPlan)
• Write Options (Map[String, String])
• isByName flag
• Write

AppendData is created using byName and byPosition operators.

isByName flag¶

AppendData is given isByName flag when created:

• byName with the flag enabled (true)
• byPosition with the flag disabled (false)

isByName is part of the V2WriteCommand abstraction.

byName¶

byName(
  table: NamedRelation,
  df: LogicalPlan,
  writeOptions: Map[String, String] = Map.empty): AppendData

byName creates a AppendData with the isByName flag enabled (true).

byName is used when:

• DataFrameWriter is requested to saveInternal (with SaveMode.Append mode) and saveAsTable (with SaveMode.Append mode)
• DataFrameWriterV2 is requested to append

byPosition¶

byPosition(
  table: NamedRelation,
  query: LogicalPlan,
  writeOptions: Map[String, String] = Map.empty): AppendData

byPosition creates a AppendData with the isByName flag disabled (false).

byPosition is used when:

• ResolveInsertInto logical resolution rule is executed
• DataFrameWriter is requested to insertInto

Execution Planning¶

AppendData is planned as one of the physical operators by DataSourceV2Strategy execution planning strategy:

• AppendDataExecV1
• AppendDataExec