GenerateSymlinkManifest (And GenerateSymlinkManifestImpl)

GenerateSymlinkManifest is a concrete post-commit hook to generate incremental and full Hive-style manifests for delta tables.

You can generate a full Hive-style manifest for delta tables using GENERATE SQL command or DeltaTable.generate operator.

GenerateSymlinkManifestImpl is a post-commit hook that…​FIXME

generateFullManifest Method

  spark: SparkSession,
  deltaLog: DeltaLog): Unit


generateFullManifest is used when…​FIXME

generateIncrementalManifest Method

  spark: SparkSession,
  deltaLog: DeltaLog,
  txnReadSnapshot: Snapshot,
  actions: Seq[Action]): Unit


generateIncrementalManifest is used when…​FIXME

Running Post-Commit Hook — run Method

  spark: SparkSession,
  txn: OptimisticTransactionImpl,
  committedActions: Seq[Action]): Unit
run is part of the PostCommitHook contract to execute a post-commit hook.

run simply generates an incremental manifest for the deltaLog and snapshot of the delta table (of the given OptimisticTransactionImpl) and the committed actions.

Handling Errors — handleError Method

  error: Throwable,
  version: Long): Unit
handleError is part of the PostCommitHook contract to handle errors while running the hook