UCSingleCatalog¶

UCSingleCatalog is a TableCatalog (Spark SQL).

UCSingleCatalog supports loading tables from the following cloud object storages:

Cloud Object Storage	Scheme
Amazon S3	`s3://`
Google Cloud Storage	`gs://`

DeltaCatalog¶

UCSingleCatalog creates a DeltaCatalog (Delta Lake) when requested to initialize.

DeltaCatalog is a DelegatingCatalogExtension (Spark SQL) that is supposed to delegate to a UCProxy.

Initialize TableCatalog¶

CatalogPlugin

initialize(
  name: String,
  options: CaseInsensitiveStringMap): Unit

initialize is part of the CatalogPlugin (Spark SQL) abstraction.

initialize creates a UCProxy to initialize.

initialize creates a DeltaCatalog (Delta Lake) that is told to delegate to the UCProxy for unsupported TableCatalog features.

Dynamic Loading

initialize expects that org.apache.spark.sql.delta.catalog.DeltaCatalog class is available on the CLASSPATH only (not at build time) and loads it dynamically.