GenerateSymlinkManifest (GenerateSymlinkManifestImpl)¶

GenerateSymlinkManifest is a post-commit hook to generate incremental and full Hive-style manifests for delta tables.

GenerateSymlinkManifest is registered when OptimisticTransactionImpl is requested to commit (with delta.compatibility.symlinkFormatManifest.enabled table property enabled).

Executing Post-Commit Hook¶

run(
  spark: SparkSession,
  txn: OptimisticTransactionImpl,
  committedActions: Seq[Action]): Unit

run is part of the PostCommitHook abstraction.

run generates an incremental manifest for the committed actions (the deltaLog and snapshot are from the OptimisticTransactionImpl).

generateIncrementalManifest¶

generateIncrementalManifest(
  spark: SparkSession,
  deltaLog: DeltaLog,
  txnReadSnapshot: Snapshot,
  actions: Seq[Action]): Unit

generateIncrementalManifest...FIXME

generateFullManifest¶

generateFullManifest(
  spark: SparkSession,
  deltaLog: DeltaLog): Unit

generateFullManifest...FIXME

generateFullManifest is used when:

• GenerateSymlinkManifestImpl is requested to generateIncrementalManifest
• DeltaGenerateCommand is executed (with symlink_format_manifest mode)