MetadataAttribute¶

MetadataAttribute is an AttributeReference with __metadata_col internal metadata.

Creating Metadata AttributeReference¶

apply(
  name: String,
  dataType: DataType,
  nullable: Boolean = true): AttributeReference

apply creates an AttributeReference with __metadata_col metadata (and true value).

Destructuring Metadata AttributeReference¶

unapply(
  attr: AttributeReference): Option[AttributeReference]

unapply returns the given AttributeReference if contains __metadata_col metadata key with true value. Otherwise, unapply returns None (an undefined value).

unapply is used when:

• FileSourceMetadataAttribute is requested to destructure an AttributeReference
• ReplaceData is requested for the dataInput

isValid¶

isValid(
  metadata: Metadata): Boolean

isValid holds true when the given Metadata contains the __metadata_col entry.

isValid is used when:

• RewriteMergeIntoTable logical rule is requested to buildWriteDeltaMergeRowsPlan
• RewriteUpdateTable logical rule is requested to buildReplaceDataUpdateProjection and buildDeletesAndInserts
• MetadataAttribute is requested to unapply
• FileSourceMetadataAttribute is requested to isValid
• MetadataColumnHelper is requested to isMetadataCol