AddFile¶
AddFile
is a FileAction that represents an action of adding a file to a delta table.
Creating Instance¶
AddFile
takes the following to be created:
- Path
- Partition values (
Map[String, String]
) - Size (in bytes)
- Modification time
-
dataChange
flag - Stats (default:
null
)
AddFile
is created when:
-
ConvertToDeltaCommand is executed (for every data file to import)
-
DelayedCommitProtocol
is requested to commit a task (after successful write) (for optimistic transactional writers)
Converting to SingleAction¶
wrap: SingleAction
wrap
is part of the Action abstraction.
wrap
creates a new SingleAction with the add
field set to this AddFile
.
Converting to RemoveFile with Defaults¶
remove: RemoveFile
remove
simply creates a RemoveFile for the path (with the current time and dataChange
flag enabled).
remove
is used when:
-
MergeIntoCommand is executed
-
WriteIntoDelta
is requested to write (withOverwrite
mode) -
DeltaSink
is requested to add a streaming micro-batch (forComplete
output mode)
Converting to RemoveFile¶
removeWithTimestamp(
timestamp: Long = System.currentTimeMillis(),
dataChange: Boolean = true): RemoveFile
remove
creates a new RemoveFile for the path with the given timestamp
and dataChange
flag.
removeWithTimestamp
is used when:
AddFile
is requested to create a RemoveFile action with the defaults- CreateDeltaTableCommand, DeleteCommand and UpdateCommand commands are executed
DeltaCommand
is requested to removeFilesFromPaths