GenerateRowIDs Logical Plan Normalization Rule¶

GenerateRowIDs is a logical rule (Rule[LogicalPlan]) that Delta Lake injects into SparkSession (using DeltaSparkSessionExtension).

Executing Rule¶

Rule

apply(
  plan: LogicalPlan): LogicalPlan

apply is part of the Rule (Spark SQL) abstraction.

apply transforms the scans on delta tables with Row Tracking enabled in the given LogicalPlan (Spark SQL) bottom-up.

apply...FIXME

metadataWithRowTrackingColumnsProjection(
  metadata: AttributeReference): NamedExpression

metadataWithRowTrackingColumnsProjection...FIXME

rowIdExpr(
  metadata: AttributeReference): Expression

rowIdExpr creates a Coalesce expression with the following expressions:

row_id (sub)attribute of the given AttributeReference
```
_metadata.row_id
```
Add expression of the base_row_id and row_index attributes of the given AttributeReference
```
_metadata.base_row_id + _metadata.row_index
```