AddFile¶

AddFile is a FileAction that represents an action of adding a file to a delta table.

Creating Instance¶

AddFile takes the following to be created:

• Path
• Partition values (Map[String, String])
• Size (in bytes)
• Modification time
• dataChange flag
• Stats (default: null)
• Tags (Map[String, String]) (default: null)

AddFile is created when:

• ConvertToDeltaCommand is executed (for every data file to import)

• DelayedCommitProtocol is requested to commit a task (after successful write) (for optimistic transactional writers)

Converting to SingleAction¶

wrap: SingleAction

wrap is part of the Action abstraction.

wrap creates a new SingleAction with the add field set to this AddFile.

Converting to RemoveFile with Defaults¶

remove: RemoveFile

remove simply creates a RemoveFile for the path (with the current time and dataChange flag enabled).

remove is used when:

• MergeIntoCommand is executed

• WriteIntoDelta is requested to write (with Overwrite mode)

• DeltaSink is requested to add a streaming micro-batch (for Complete output mode)

Converting to RemoveFile¶

removeWithTimestamp(
  timestamp: Long = System.currentTimeMillis(),
  dataChange: Boolean = true): RemoveFile

remove creates a new RemoveFile for the path with the given timestamp and dataChange flag.

removeWithTimestamp is used when:

• AddFile is requested to create a RemoveFile action with the defaults
• CreateDeltaTableCommand, DeleteCommand and UpdateCommand commands are executed
• DeltaCommand is requested to removeFilesFromPaths