CreateDeltaTableCommand¶

CreateDeltaTableCommand is a leaf RunnableCommand (Spark SQL) to create a delta table (for DeltaCatalog).

Creating Instance¶

CreateDeltaTableCommand takes the following to be created:

• CatalogTable (Spark SQL)
• Existing CatalogTable (if available)
• SaveMode
• Optional Data Query (LogicalPlan)
• CreationMode
• tableByPath flag (default: false)
• Output attributes (default: empty)

CreateDeltaTableCommand is created when:

• DeltaCatalog is requested to create a delta table

CreationMode¶

CreateDeltaTableCommand can be given a CreationMode when created:

• Create (default)
• CreateOrReplace
• Replace

CreationMode is Create by default or specified by DeltaCatalog.

Executing Command¶

run(
  sparkSession: SparkSession): Seq[Row]

run is part of the RunnableCommand (Spark SQL) abstraction.

run...FIXME

updateCatalog¶

updateCatalog(
  spark: SparkSession,
  table: CatalogTable): Unit

updateCatalog uses the given SparkSession to access SessionCatalog to createTable or alterTable when the tableByPath flag is off. Otherwise, updateCatalog does nothing.

getOperation¶

getOperation(
  metadata: Metadata,
  isManagedTable: Boolean,
  options: Option[DeltaOptions]): DeltaOperations.Operation

getOperation...FIXME

replaceMetadataIfNecessary¶

replaceMetadataIfNecessary(
  txn: OptimisticTransaction,
  tableDesc: CatalogTable,
  options: DeltaOptions,
  schema: StructType): Unit

replaceMetadataIfNecessary determines whether or not it is a replace operation (i.e., CreateOrReplace or Replace based on the CreationMode).

replaceMetadataIfNecessary determines whether or not it is supposed not to overwrite the schema of a Delta table (based on the overwriteSchema option in the input DeltaOptions).

In the end, only for an CreateOrReplace or Replace operation on an existing delta table with overwriteSchema option enabled, replaceMetadataIfNecessary updates the metadata (on the given OptimisticTransaction) with the given schema.

DeltaIllegalArgumentException¶

replaceMetadataIfNecessary throws an DeltaIllegalArgumentException for a CreateOrReplace or Replace operation with overwriteSchema option enabled:

The usage of overwriteSchema is not allowed when replacing a Delta table.