Skip to content

DeltaCatalog

DeltaCatalog is an extension of Spark SQL (using DelegatingCatalogExtension and StagingTableCatalog).

DeltaCatalog is registered using spark.sql.catalog.spark_catalog configuration property (while creating a SparkSession in a Spark application).

== [[alterTable]] Altering Table

[source,scala]

alterTable( ident: Identifier, changes: TableChange*): Table


alterTable...FIXME

alterTable is part of the TableCatalog (Spark SQL 3.0.0) abstraction.

Creating Table

createTable(
  ident: Identifier,
  schema: StructType,
  partitions: Array[Transform],
  properties: util.Map[String, String]): Table

createTable...FIXME

createTable is part of the TableCatalog (Spark SQL) abstraction.

== [[loadTable]] Loading Table

[source,scala]

loadTable( ident: Identifier): Table


loadTable...FIXME

loadTable is part of the TableCatalog (Spark SQL 3.0.0) abstraction.

Creating Delta Table

createDeltaTable(
  ident: Identifier,
  schema: StructType,
  partitions: Array[Transform],
  properties: util.Map[String, String],
  sourceQuery: Option[LogicalPlan],
  operation: TableCreationModes.CreationMode): Table

createDeltaTable...FIXME

createDeltaTable is used when:


Last update: 2020-09-24