OverwritePartitionsDynamic Logical Command¶

OverwritePartitionsDynamic is a V2WriteCommand for dynamically overwrite partitions in an existing table (that supports dynamic overwrite in batch mode).

Creating Instance¶

OverwritePartitionsDynamic takes the following to be created:

• NamedRelation
• Query
• Write Options
• isByName flag

OverwritePartitionsDynamic is created (indirectly) using byName and byPosition utilities.

Creating OverwritePartitionsDynamic by Name¶

byName(
  table: NamedRelation,
  df: LogicalPlan,
  writeOptions: Map[String, String] = Map.empty): OverwritePartitionsDynamic

byName creates a OverwritePartitionsDynamic with the isByName flag enabled (true).

byName is used when:

• DataFrameWriterV2 is requested to overwritePartitions

Creating OverwritePartitionsDynamic by Position¶

byPosition(
  table: NamedRelation,
  query: LogicalPlan,
  writeOptions: Map[String, String] = Map.empty): OverwritePartitionsDynamic

byPosition creates a OverwritePartitionsDynamic with the isByName flag disabled (false).

byPosition is used when:

• ResolveInsertInto logical resolution rule is executed (for InsertIntoStatements over DataSourceV2Relation)
• DataFrameWriter is requested to insertInto

Execution Planning¶

OverwritePartitionsDynamic (over DataSourceV2Relation) is planned as OverwritePartitionsDynamicExec physical operator by DataSourceV2Strategy execution planning strategy.

TableCapabilityCheck¶

TableCapabilityCheck extended analysis check asserts that OverwritePartitionsDynamic uses DataSourceV2Relation that supports dynamic overwrite in batch mode.