Metadata¶

Metadata is an Action to update the metadata of a delta table (indirectly via the Snapshot).

Use DescribeDeltaDetailCommand to review the metadata of a delta table.

Creating Instance¶

Metadata takes the following to be created:

• Id (default: random UUID)
• Name (default: null)
• Description (default: null)
• Format (default: empty)
• Schema (default: null)
• Partition Columns (default: Nil)
• Table Configuration (default: Map.empty)
• Created Time (default: current time)

Metadata is created when:

• DeltaLog is requested for the metadata (but that should be rare)
• InitialSnapshot is created
• ConvertToDeltaCommand is executed
• ImplicitMetadataOperation is requested to updateMetadata

Updating Metadata¶

Metadata can be updated in a transaction once only (and only when created for an uninitialized table, when readVersion is -1).

txn.metadata

Demo¶

val path = "/tmp/delta/users"

import org.apache.spark.sql.delta.DeltaLog
val deltaLog = DeltaLog.forTable(spark, path)

import org.apache.spark.sql.delta.actions.Metadata
assert(deltaLog.snapshot.metadata.isInstanceOf[Metadata])

deltaLog.snapshot.metadata.id

Table ID¶

Metadata uses a Table ID (aka reservoirId) to uniquely identify a delta table and is never going to change through the history of the table.

Table ID is given Metadata is created or defaults to a random UUID (Java).