Skip to content

CreateDeltaTableCommand

CreateDeltaTableCommand is a Spark SQL RunnableCommand (and executed eagerly on the driver for side-effects).

Creating Instance

CreateDeltaTableCommand takes the following to be created:

  • CatalogTable (Spark SQL)
  • Existing CatalogTable (if available)
  • SaveMode
  • Optional Data Query (LogicalPlan)
  • CreationMode (default: TableCreationModes.Create)
  • tableByPath flag (default: false)

CreateDeltaTableCommand is created when DeltaCatalog is requested to create a Delta table.

Executing Command

run(
  sparkSession: SparkSession): Seq[Row]

run creates a DeltaLog (for the given table based on a table location) and a DeltaOptions.

run starts a transaction (on the DeltaLog).

run branches off based on the optional data query. For data query defined, run creates a WriteIntoDelta and requests it to write. Otherwise, run creates an empty table.

Note

run does a bit more, but I don't think it's of much interest.

run commits the transaction.

In the end, run updateCatalog.

run is part of the RunnableCommand abstraction.

Updating Catalog

updateCatalog(
  spark: SparkSession,
  table: CatalogTable): Unit

updateCatalog uses the given SparkSession to access SessionCatalog to createTable or alterTable when the tableByPath flag is off. Otherwise, updateCatalog does nothing.


Last update: 2020-09-24