FileSourceMetadataAttribute¶

Cleaning Up Metadata¶

cleanupFileSourceMetadataInformation(
  attr: Attribute): Attribute
cleanupFileSourceMetadataInformation(
  field: StructField): StructField

cleanupFileSourceMetadataInformation removes internal metadata from (the metadata of) the given Attribute or StructField.

cleanupFileSourceMetadataInformation is used when:

• FileFormatWriter is requested to write data out
• FileFormat is requested to create a FileFormat metadata struct column

removeInternalMetadata¶

removeInternalMetadata(
  metadata: Metadata)

removeInternalMetadata creates a new MetadataBuilder to build a metadata for the given Metadata but without the following metadata entries:

• __metadata_col
• __file_source_metadata_col
• __file_source_constant_metadata_col
• __file_source_generated_metadata_col

Creating Metadata AttributeReference¶

apply(
  name: String,
  dataType: DataType,
  nullable: Boolean = false): AttributeReference

apply creates an AttributeReference with the following metadata:

apply is used when:

• FileFormat is requested to createFileMetadataCol