Skip to content

DeltaOptimizeBuilder

DeltaOptimizeBuilder is a builder interface for constructing and executing OPTIMIZE command.

DeltaOptimizeBuilder is created using DeltaTable.optimize operator.

In the end, DeltaOptimizeBuilder is supposed to be executed to take action using the following operators:

io.delta.tables Package

DeltaTableBuilder belongs to io.delta.tables package.

import io.delta.tables.DeltaTableBuilder

Demo

import io.delta.tables.DeltaTable
DeltaTable.forName("part_delta")
  .optimize()
  .where("p = 0")
  .executeZOrderBy("x", "y)
  .show(truncate = false)

Operators

executeCompaction

executeCompaction(): DataFrame

executeCompaction executes this DeltaOptimizeBuilder (with no zOrderBy attributes).

executeZOrderBy

executeZOrderBy(
  columns: String *): DataFrame

executeZOrderBy executes this DeltaOptimizeBuilder (with the given columns as zOrderBy attributes).

where

where(
  partitionFilter: String): DeltaOptimizeBuilder

where registers a partitionFilter and returns this DeltaOptimizeBuilder.

Creating Instance

DeltaOptimizeBuilder takes the following to be created:

DeltaOptimizeBuilder is created using apply factory method.

Creating DeltaOptimizeBuilder

apply(
  sparkSession: SparkSession,
  tableIdentifier: String): DeltaOptimizeBuilder

apply creates a DeltaOptimizeBuilder.

A private method

apply is a private method and can only be executed using DeltaTable.optimize operator.

apply is used when:

Executing

execute(
  zOrderBy: Seq[UnresolvedAttribute]): DataFrame

execute creates an OptimizeTableCommand (with tableId, the partitionFilter and the given zOrderBy attributes) and executes it (while creating a DataFrame).

execute is used when: