DeltaConfigs — Table Property

DeltaConfigs (of type T) represents a named property of a delta table with values (of type T) in a validated range.

DeltaConfigs is persisted to the Metadata.

DeltaConfigs table property can be assigned a value using ALTER TABLE SQL command:

ALTER TABLE <table_name> SET TBLPROPERTIES (<key>=<value>)
Table 1. DeltaConfigs
Key Description


Whether this delta table will automagically optimize the layout of files during writes.

Default: false


How often to checkpoint the state of a delta table

Default: 10


How long to keep checkpoint files around before deleting them

Default: interval 2 days

The most recent checkpoint is never deleted. It is acceptable to keep checkpoint files beyond this duration until the next calendar day.


The number of columns to collect stats on for data skipping. -1 means collecting stats for all columns.

Default: 32


Whether to clean up expired log files and checkpoints

Default: true


When enabled (default), a delta table can be rolled back to any point within logRetentionDuration. When disabled, the table can be rolled back checkpointRetentionDuration only.

Default: true


Whether a delta table is append-only (true) or not (false). When enabled, a table allows appends only and no updates or deletes.

Default: false


How long to keep obsolete logs around before deleting them. Delta can keep logs beyond the duration until the next calendar day to avoid constantly creating checkpoints.

Default: interval 30 days


Whether to use a random prefix in a file path instead of partition information (may be required for very high volume S3 calls to better be partitioned across S3 servers)

Default: false


The length of the random prefix in a file path for randomizeFilePrefixes

Default: 2


How long to keep delta sample files around before deleting them

Default: interval 7 days


How long to keep logically deleted data files around before deleting them physically (to prevent failures in stale readers after compactions or partition overwrites)

Default: interval 1 week

Creating DeltaConfigs Instance

DeltaConfigs takes the following to be created:

  • Key

  • Default value

  • Conversion function (from text representation of the DeltaConfigs to the type, i.e. String ⇒ T)

  • Validation function (that guards from incorrect values, i.e. T ⇒ Boolean)

  • Help message

  • (optional) Minimum version of protocol supported

DeltaConfigs initializes the internal properties.

Reading Configuration Property From Metadata — fromMetaData Method

fromMetaData(metadata: Metadata): T

fromMetaData looks up the key of the DeltaConfigs configuration in the configuration of the given Metadata. If not found, fromMetaData takes the default value.

In the end, fromMetaData converts the text representation to the proper type using fromString conversion function.

fromMetaData is used when:

mergeGlobalConfigs Utility

  sqlConfs: SQLConf,
  tableConf: Map[String, String],
  protocol: Protocol): Map[String, String]


mergeGlobalConfigs is used when…​FIXME