Skip to content

DeltaGenerateCommand

DeltaGenerateCommand is a RunnableCommand (Spark SQL) to execute a generate function on a delta table.

DeltaGenerateCommand is used for the following:

DeltaGenerateCommand supports symlink_format_manifest mode name only.

Demo

val path = "/tmp/delta/d01"
val tid = s"delta.`$path`"
val q = s"GENERATE symlink_format_manifest FOR TABLE $tid"
sql(q).collect

Creating Instance

DeltaGenerateCommand takes the following to be created:

DeltaGenerateCommand is created for:

Generate Mode Name

DeltaGenerateCommand is given a mode name when created.

DeltaGenerateCommand uses a lookup table of the supported generation functions by mode name (yet supports just symlink_format_manifest).

Mode Name Generation Function
symlink_format_manifest generateFullManifest

Executing Command

run(
  sparkSession: SparkSession): Seq[Row]

run is part of the RunnableCommand (Spark SQL) abstraction.

run creates a Hadoop Path to (the location of) the delta table (based on DeltaTableIdentifier).

run creates a DeltaLog for the delta table.

run executes the generation function for the mode name.

run returns no rows (an empty collection).

IllegalArgumentException

run throws an IllegalArgumentException when executed with an unsupported mode name:

Specified mode '[modeName]' is not supported. Supported modes are: [supportedModes]

AnalysisException

run throws an AnalysisException when executed for a non-delta table:

GENERATE is only supported for Delta tables.

Last update: 2021-03-19