Catalogs¶

Catalogs is a utility to load (and initialize) a CatalogPlugin by a given name.

Loading Catalog by Name¶

load(
  name: String,
  conf: SQLConf): CatalogPlugin

load finds the class name of the CatalogPlugin in the given SQLConf by spark.sql.catalog.[name] key (or throws an CatalogNotFoundException).

load loads the class and makes sure that it is a CatalogPlugin (or throws an SparkException).

In the end, load creates a new instance (using a public no-arg constructor) and requests the CatalogPlugin to initialize (with the given name and all the catalog options that use spark.sql.catalog.[name] prefix).

load is used when:

• CatalogManager is requested to look up a catalog by name and loadV2SessionCatalog

CatalogNotFoundException¶

load throws a CatalogNotFoundException when the spark.sql.catalog.[name] key could not be found:

Catalog '[name]' plugin class not found: spark.sql.catalog.[name] is not defined

SparkException¶

load throws a SparkException when the class name is not of the CatalogPlugin type:

Plugin class for catalog '[name]' does not implement CatalogPlugin: [pluginClassName]

Collecting Catalog Options¶

catalogOptions(
  name: String,
  conf: SQLConf): CaseInsensitiveStringMap

catalogOptions collects all options with spark.sql.catalog.[name]. prefix (in the given SQLConf).