DescribeDeltaDetailCommand¶

DescribeDeltaDetailCommand is a leaf RunnableCommand (Spark SQL) that represents the following high-level operators at execution:

DESCRIBE DETAIL SQL command
DeltaTable.detail

Creating Instance¶

DescribeDeltaDetailCommand takes the following to be created:

Child logical operator (Spark SQL)
Hadoop Configuration (Map[String, String])

DescribeDeltaDetailCommand is created using apply utility.

Create DescribeDeltaDetailCommand¶

apply(
  path: Option[String],
  tableIdentifier: Option[TableIdentifier],
  hadoopConf: Map[String, String]): DescribeDeltaDetailCommand

apply creates a UnresolvedPathOrIdentifier for the optional path and tableIdentifier.

In the end, apply creates a DescribeDeltaDetailCommand.

apply is used when:

DeltaSqlAstBuilder is requested to parse DESCRIBE DETAIL SQL command
DeltaTableOperations is requested to executeDetails

Output (Schema) Attributes¶

QueryPlan

output: Seq[Attribute]

output is part of the QueryPlan (Spark SQL) abstraction.

output is the fields of TableDetail.

Execute Command¶

RunnableCommand

run(
  sparkSession: SparkSession): Seq[Row]

run is part of the RunnableCommand (Spark SQL) abstraction.

run finds the CatalogTable (of this table).

run...FIXME

For an existing delta table (the version is at least 0), run describeDeltaTable.

describeDeltaTable¶

describeDeltaTable(
  sparkSession: SparkSession,
  deltaLog: DeltaLog,
  snapshot: Snapshot,
  tableMetadata: Option[CatalogTable]): Seq[Row]

describeDeltaTable...FIXME

Demo¶

SQL

DESCRIBE DETAIL '/tmp/delta/users'

scala> q.show
+------+--------------------+----+-----------+--------------------+--------------------+-------------------+----------------+--------+-----------+----------+----------------+----------------+
|format|                  id|name|description|            location|           createdAt|       lastModified|partitionColumns|numFiles|sizeInBytes|properties|minReaderVersion|minWriterVersion|
+------+--------------------+----+-----------+--------------------+--------------------+-------------------+----------------+--------+-----------+----------+----------------+----------------+
| delta|3799b291-dbfa-4f8...|null|       null|file:/tmp/delta/u...|2020-01-06 17:08:...|2020-01-06 17:12:28| [city, country]|       4|       2581|        []|               1|               2|
+------+--------------------+----+-----------+--------------------+--------------------+-------------------+----------------+--------+-----------+----------+----------------+----------------+