CatalogManager¶

Creating Instance¶

CatalogManager takes the following to be created:

• SQLConf
• Default Session Catalog
• SessionCatalog

CatalogManager is created when:

• BaseSessionStateBuilder is requested for a CatalogManager

Default Session Catalog¶

defaultSessionCatalog: CatalogPlugin

CatalogManager is given a CatalogPlugin when created for the default session catalog.

defaultSessionCatalog is used as the delegate catalog when CatalogManager is requested to load a V2SessionCatalog.

defaultSessionCatalog is used as the fall-back catalog when CatalogManager is requested to load a custom V2CatalogPlugin.

Default Catalog Name¶

CatalogManager defines spark_catalog as the name of the default catalog (V2SessionCatalog).

spark_catalog is used as the default value of spark.sql.defaultCatalog configuration property.

Current Catalog Name¶

_currentCatalogName: Option[String]

_currentCatalogName is the name of the current catalog and is undefined by default (None).

_currentCatalogName can be changed using setCurrentCatalog.

Current CatalogPlugin¶

currentCatalog: CatalogPlugin

currentCatalog uses the current CatalogPlugin if defined or falls back on spark.sql.defaultCatalog configuration property.

currentCatalog is used when:

• CatalogManager is requested for the current namespace, setCurrentNamespace or setCurrentCatalog

• LookupCatalog is requested to currentCatalog

• ViewHelper utility is requested to generateViewProperties

Current Namespace¶

currentNamespace: Array[String]

currentNamespace...FIXME

currentNamespace is used when:

• ResolveNamespace analysis rule is executed
• GetCurrentDatabase analysis rule is executed
• CatalogAndIdentifier extractor utility is requested to unapply
• ViewHelper utility is requested to generateViewProperties

Setting Current Namespace¶

setCurrentNamespace(
  namespace: Array[String]): Unit

setCurrentNamespace...FIXME

setCurrentNamespace is used when SetCatalogAndNamespaceExec physical command is executed.

Changing Current Catalog Name¶

setCurrentCatalog(
  catalogName: String): Unit

setCurrentCatalog checks out whether the given catalog name is different from the currentCatalog's.

Only if the names are different, setCurrentCatalog makes it _currentCatalogName and "resets" _currentNamespace (None). In the end, setCurrentCatalog requests the SessionCatalog to setCurrentDatabase as default.

setCurrentCatalog is used when:

• SetCatalogCommand logical command is executed
• SetCatalogAndNamespaceExec physical command is executed

Finding CatalogPlugin by Name¶

catalog(
  name: String): CatalogPlugin

catalog returns the v2 session catalog when the given name is spark_catalog.

Otherwise, catalog looks up the name in catalogs internal registry. When not found, catalog tries to load a CatalogPlugin by name (and registers it in catalogs internal registry).

catalog is used when:

• CatalogManager is requested to isCatalogRegistered and currentCatalog

• CatalogV2Util utility is requested to getTableProviderCatalog

• CatalogAndMultipartIdentifier, CatalogAndNamespace and CatalogAndIdentifier utilities are requested to extract a CatalogPlugin (unapply)

isCatalogRegistered¶

isCatalogRegistered(
  name: String): Boolean

isCatalogRegistered...FIXME

isCatalogRegistered is used when Analyzer is requested to expandRelationName.

v2SessionCatalog¶

v2SessionCatalog: CatalogPlugin

v2SessionCatalog...FIXME

v2SessionCatalog is used when:

• CatalogManager is requested to look up a CatalogPlugin by name

• CatalogV2Util is requested to getTableProviderCatalog

• CatalogAndIdentifier utility is requested to extract a CatalogPlugin and an identifier from a multi-part name (unapply)

loadV2SessionCatalog¶

loadV2SessionCatalog(): CatalogPlugin

loadV2SessionCatalog loads the default spark_catalog.

If it is of type CatalogExtension, loadV2SessionCatalog requests it to setDelegateCatalog with the defaultSessionCatalog.

loadV2SessionCatalog is used when:

• CatalogManager is requested for a CatalogPlugin