Metadata Columns¶

Spark 3.1.1 (SPARK-31255) introduced support for MetadataColumns for additional metadata of a row.

MetadataColumns can be defined for Tables with SupportsMetadataColumns.

Use DESCRIBE TABLE EXTENDED SQL command to display the metadata columns of a table.

DataSourceV2Relation¶

MetadataColumns are disregarded (filtered out) from the metadataOutput in DataSourceV2Relation leaf logical operator when in name-conflict with output columns.