MetadataAttribute¶

Creating Metadata AttributeReference¶

apply(
  name: String,
  dataType: DataType,
  nullable: Boolean = true): AttributeReference

apply creates an AttributeReference with __metadata_col metadata (and true value).

Destructuring Metadata AttributeReference¶

unapply(
  attr: AttributeReference): Option[AttributeReference]

unapply returns the given AttributeReference if contains __metadata_col metadata key with true value. Otherwise, unapply returns None (an undefined value).

unapply is used when:

• FileSourceMetadataAttribute is requested to destructure an AttributeReference
• ReplaceData is requested for the dataInput