UCProxy¶

UCProxy is aTableCatalog (Spark SQL).

TablesApi¶

UCProxy creates a TablesApi client when requested to initialize.

UCProxy creates a TemporaryTableCredentialsApi client when requested to initialize.

CatalogPlugin

initialize(
  name: String,
  options: CaseInsensitiveStringMap): Unit

initialize is part of the CatalogPlugin (Spark SQL) abstraction.

initialize asserts that uri option is specified.

initialize creates an ApiClient based on the uri option (the host, the port and the scheme).

initialize tells the ApiClient to use Authorization header for every request based on token option, if defined.

In the end, initialize creates this TablesApi and this TemporaryTableCredentialsApi.

TableCatalog

loadTable(
  ident: Identifier): Table

loadTable is part of the TableCatalog (Spark SQL) abstraction.

loadTable requests the TablesApi to load the metadata of the table by the fully-qualified table name (with the name of this catalog):

[catalog_name].[two-part ident (with the schema and table names)]

If the table metadata could be found, loadTable requests the TemporaryTableCredentialsApi for the S3 temporary table credentials (sending a POST request to /temporary-table-credentials endpoint).

Note

It is assumed that the temporary table credentials are for READ_WRITE operation.

For a table stored in a S3 bucket, loadTable sets the following Hadoop properties:

Hadoop Property	Value
`fs.s3a.access.key`	`accessKeyId` of the `AwsCredentials`
`fs.s3a.secret.key`	`secretAccessKey` of the `AwsCredentials`
`fs.s3a.session.token`	`sessionToken` of the `AwsCredentials`
`fs.s3a.impl`	`org.apache.hadoop.fs.s3a.S3AFileSystem`
`fs.s3a.path.style.access`	`true`

loadTable creates a CatalogTable (Spark SQL) with the following:

In the end, loadTable creates a V1Table (Spark SQL) for the CatalogTable.

SupportsNamespaces

listNamespaces(): Array[Array[String]]

listNamespaces is part of the SupportsNamespaces (Spark SQL) abstraction.

listNamespaces...FIXME

TableCatalog

listTables(
  namespace: Array[String]): Array[Identifier]

listTables is part of the TableCatalog (Spark SQL) abstraction.

listTables...FIXME