Skip to content

CreateDeltaTableCommand

CreateDeltaTableCommand is a leaf RunnableCommand (Spark SQL) to create a delta table (for DeltaCatalog).

Creating Instance

CreateDeltaTableCommand takes the following to be created:

  • CatalogTable (Spark SQL)
  • Existing CatalogTable (if available)
  • SaveMode
  • Optional Data Query (LogicalPlan)
  • CreationMode
  • tableByPath flag (default: false)
  • Output attributes (default: empty)

CreateDeltaTableCommand is created when:

CreationMode

CreateDeltaTableCommand can be given a CreationMode when created:

  • Create (default)
  • CreateOrReplace
  • Replace

CreationMode is Create by default or specified by DeltaCatalog.

Executing Command

run(
  sparkSession: SparkSession): Seq[Row]

run is part of the RunnableCommand (Spark SQL) abstraction.


run...FIXME

updateCatalog

updateCatalog(
  spark: SparkSession,
  table: CatalogTable): Unit

updateCatalog uses the given SparkSession to access SessionCatalog to createTable or alterTable when the tableByPath flag is off. Otherwise, updateCatalog does nothing.

getOperation

getOperation(
  metadata: Metadata,
  isManagedTable: Boolean,
  options: Option[DeltaOptions]): DeltaOperations.Operation

getOperation...FIXME

replaceMetadataIfNecessary

replaceMetadataIfNecessary(
  txn: OptimisticTransaction,
  tableDesc: CatalogTable,
  options: DeltaOptions,
  schema: StructType): Unit

Unused argument

tableDesc argument is not used.

replaceMetadataIfNecessary determines whether or not it is a replace operation (i.e., CreateOrReplace or Replace based on the CreationMode).

replaceMetadataIfNecessary determines whether or not it is supposed not to overwrite the schema of a Delta table (based on the overwriteSchema option in the input DeltaOptions).

In the end, only for an CreateOrReplace or Replace operation on an existing delta table with overwriteSchema option enabled, replaceMetadataIfNecessary updates the metadata (on the given OptimisticTransaction) with the given schema.

DeltaIllegalArgumentException

replaceMetadataIfNecessary throws an DeltaIllegalArgumentException for a CreateOrReplace or Replace operation with overwriteSchema option enabled:

The usage of overwriteSchema is not allowed when replacing a Delta table.