ImplicitMetadataOperation

= [[ImplicitMetadataOperation]] ImplicitMetadataOperation -- Operations Updating Metadata (Schema And Partitioning)

ImplicitMetadataOperation is an <> of <> that can <> of a delta table (while writing out a new data to a delta table).

ImplicitMetadataOperation operations can update schema by <> and <> schema.

[[contract]] .ImplicitMetadataOperation Contract (Abstract Methods Only) [cols="30m,70",options="header",width="100%"] |=== | Method | Description

| canMergeSchema a| [[canMergeSchema]]

[source, scala]¶

canMergeSchema: Boolean¶

Used when ImplicitMetadataOperation is requested to <>

| canOverwriteSchema a| [[canOverwriteSchema]]

[source, scala]¶

canOverwriteSchema: Boolean¶

Used when ImplicitMetadataOperation is requested to <>

|===

[[implementations]] .ImplicitMetadataOperations [cols="30,70",options="header",width="100%"] |=== | ImplicitMetadataOperation | Description

| <> | [[WriteIntoDelta]] Delta command for batch queries (Spark SQL)

| <> | [[DeltaSink]] Streaming sink for streaming queries (Spark Structured Streaming)

|===

Updating Metadata¶

updateMetadata(
  txn: OptimisticTransaction,
  data: Dataset[_],
  partitionColumns: Seq[String],
  configuration: Map[String, String],
  isOverwriteMode: Boolean): Unit

updateMetadata...FIXME

updateMetadata is used when:

• WriteIntoDelta command is executed
• DeltaSink is requested to add a streaming micro-batch

== [[normalizePartitionColumns]] Normalize Partition Columns -- normalizePartitionColumns Internal Method

[source, scala]¶

normalizePartitionColumns( spark: SparkSession, partitionCols: Seq[String], schema: StructType): Seq[String]

normalizePartitionColumns...FIXME

NOTE: normalizePartitionColumns is used when ImplicitMetadataOperation is requested to <>.