Skip to content

TemporaryTableCredentialsService

TemporaryTableCredentialsService is a Unity Catalog API service that UnityCatalogServer uses to handle HTTP requests at /api/2.1/unity-catalog/temporary-table-credentials URL.

Method URL Handler Params
POST generateTemporaryTableCredential GenerateTemporaryTableCredential

TableRepository

TemporaryTableCredentialsService gets at the system-wide TableRepository instance when created.

Looking Up Table Credentials

HttpResponse generateTemporaryTableCredential(
  GenerateTemporaryTableCredential generateTemporaryTableCredential)

generateTemporaryTableCredential looks up the table metadata (by the tableId of the given GenerateTemporaryTableCredential) in the TableRepository.

Unless the storage location is on S3 (starts with s3:// prefix), generateTemporaryTableCredential assumes local file system and returns empty credentials.

For a S3 storage location, generateTemporaryTableCredential finds the S3 storage location credentials.

Demo

Local File System

$ http http://localhost:8081/api/2.1/unity-catalog/temporary-table-credentials \
    table_id=b5d6db68-5eca-485c-be5f-5f53d4f27f60 \
    operation=READ
HTTP/1.1 200 OK
content-length: 52
content-type: application/json
date: Tue, 9 Jul 2024 11:54:58 GMT
server: Armeria/1.28.4

{
    "aws_temp_credentials": null,
    "expiration_time": null
}

S3 Storage Location

Register a table on S3.

$ ./bin/uc table create \
    --full_name unity.default.s3_creds_demo \
    --columns "id INT, name STRING" \
    --storage_location "s3://japila.pl/my_demo_bucket/s3_creds_demo"
┌────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
│        KEY         │                                               VALUE                                                │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│NAME                │s3_creds_demo                                                                                       │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│CATALOG_NAME        │unity                                                                                               │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│SCHEMA_NAME         │default                                                                                             │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│TABLE_TYPE          │EXTERNAL                                                                                            │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│DATA_SOURCE_FORMAT  │DELTA                                                                                               │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│COLUMNS             │{"name":"id","type_text":"int","type_json":"{\"name\":\"id\",\"type\":\"integer\",\"nullable\":true,│
│                    │\"metadata\":{}}","type_name":"INT","type_precision":0,"type_scale":0,"type_interval_type":null,"pos│
│                    │ition":0,"comment":null,"nullable":true,"partition_index":null}                                     │
│                    │{"name":"name","type_text":"string","type_json":"{\"name\":\"name\",\"type\":\"string\",\"nullable\"│
│                    │:true,\"metadata\":{}}","type_name":"STRING","type_precision":0,"type_scale":0,"type_interval_type":│
│                    │null,"position":1,"comment":null,"nullable":true,"partition_index":null}                            │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│STORAGE_LOCATION    │s3://japila.pl/my_demo_bucket/s3_creds_demo                                                         │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│COMMENT             │null                                                                                                │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│PROPERTIES          │{}                                                                                                  │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│CREATED_AT          │1720539043045                                                                                       │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│UPDATED_AT          │null                                                                                                │
├────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│TABLE_ID            │020270b2-621e-4157-a73d-6a63c3b86bdd                                                                │
└────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
$ http http://localhost:8081/api/2.1/unity-catalog/temporary-table-credentials \
    table_id=020270b2-621e-4157-a73d-6a63c3b86bdd   \
    operation=READ

HTTP/1.1 400 Bad Request
content-length: 192
content-type: application/json
date: Tue, 9 Jul 2024 14:54:48 GMT
server: Armeria/1.28.4

{
    "details": [
        {
            "@type": "google.rpc.ErrorInfo",
            "metadata": {},
            "reason": "FAILED_PRECONDITION"
        }
    ],
    "error_code": "FAILED_PRECONDITION",
    "message": "S3 bucket configuration not found.",
    "stack_trace": null
}

Add a S3 bucket configuration to etc/conf/server.properties for the table location and bounce the UC server.

$ cat etc/conf/server.properties
server.env=dev
## temp credential config for s3 (Multiple s3 config can be added by incrementing the index)
s3.bucketPath.0=s3://japila.pl
s3.accessKey.0=FIXME_accessKey
s3.secretKey.0=FIXME_secretKey
s3.sessionToken.0=FIXME_sessionToken
./bin/start-uc-server
$ http http://localhost:8081/api/2.1/unity-catalog/temporary-table-credentials \
    table_id=020270b2-621e-4157-a73d-6a63c3b86bdd   \
    operation=READ
HTTP/1.1 200 OK
content-length: 158
content-type: application/json
date: Tue, 9 Jul 2024 15:32:04 GMT
server: Armeria/1.28.4

{
    "aws_temp_credentials": {
        "access_key_id": "FIXME_accessKey",
        "secret_access_key": "FIXME_secretKey",
        "session_token": "FIXME_sessionToken"
    },
    "expiration_time": null
}