Skip to content

GenerateSymlinkManifest (GenerateSymlinkManifestImpl)

GenerateSymlinkManifest is a post-commit hook to generate incremental and full Hive-style manifests for delta tables.

GenerateSymlinkManifest is registered when OptimisticTransactionImpl is requested to commit (with delta.compatibility.symlinkFormatManifest.enabled table property enabled).

Executing Post-Commit Hook

run(
  spark: SparkSession,
  txn: OptimisticTransactionImpl,
  committedActions: Seq[Action]): Unit

run is part of the PostCommitHook abstraction.

run generates an incremental manifest for the committed actions (the deltaLog and snapshot are from the OptimisticTransactionImpl).

generateIncrementalManifest

generateIncrementalManifest(
  spark: SparkSession,
  deltaLog: DeltaLog,
  txnReadSnapshot: Snapshot,
  actions: Seq[Action]): Unit

generateIncrementalManifest...FIXME

generateFullManifest

generateFullManifest(
  spark: SparkSession,
  deltaLog: DeltaLog): Unit

generateFullManifest...FIXME

generateFullManifest is used when: