Skip to content

AddFile

AddFile is a <> to denote a <> added to a <>.

AddFile is <> when:

  • <> is executed (for <>)

  • DelayedCommitProtocol is requested to <> (for <>)

== [[creating-instance]] Creating AddFile Instance

AddFile takes the following to be created:

  • [[path]] Path
  • [[partitionValues]] Partition values (Map[String, String])
  • [[size]] Size (in bytes)
  • [[modificationTime]] Modification time
  • [[dataChange]] dataChange flag
  • [[stats]] Stats (default: null)
  • [[tags]] Tags (Map[String, String]) (default: null)

== [[wrap]] wrap Method

[source, scala]

wrap: SingleAction

NOTE: wrap is part of the <> contract to wrap the action into a <> for serialization.

wrap simply creates a new <> with the add field set to this AddFile.

== [[remove]] Creating RemoveFile Instance With Current Timestamp -- remove Method

[source, scala]

remove: RemoveFile

remove <> with the current timestamp and dataChange flag enabled.

[NOTE]

remove is used when:

  • <> is executed

  • <> is executed (with Overwrite mode)

* DeltaSink is requested to <> (with Complete mode)

== [[removeWithTimestamp]] Creating RemoveFile Instance For Given Timestamp -- removeWithTimestamp Method

[source, scala]

removeWithTimestamp( timestamp: Long = System.currentTimeMillis(), dataChange: Boolean = true): RemoveFile


removeWithTimestamp creates a <> action for the <>, and the given timestamp and dataChange flag.

[NOTE]

removeWithTimestamp is used when:

  • AddFile is requested to <>

* <> and <> are executed


Last update: 2020-09-29