Skip to content

CommitInfo

CommitInfo is an Action defined by the following properties:

  • Version (optional)
  • Timestamp
  • User ID (optional)
  • User Name (optional)
  • Operation
  • Operation Parameters
  • JobInfo (optional)
  • NotebookInfo (optional)
  • Cluster ID (optional)
  • Read Version (optional)
  • Isolation Level (optional)
  • isBlindAppend flag (optional)
  • Operation Metrics (optional)
  • User metadata (optional)

CommitInfo is created (using apply and empty utilities) when:

CommitInfo is used as a part of OptimisticTransactionImpl and CommitStats.

isBlindAppend flag

CommitInfo is given isBlindAppend flag when created.

isBlindAppend flag indicates whether a commit has blindly appended data without caring about existing files.

DeltaHistoryManager

CommitInfo can be looked up using DeltaHistoryManager for the following:

spark.databricks.delta.commitInfo.enabled

CommitInfo is added (logged) to a delta log only with spark.databricks.delta.commitInfo.enabled configuration property enabled.

Creating Empty CommitInfo

empty(
  version: Option[Long] = None): CommitInfo

empty...FIXME

empty is used when:

Creating CommitInfo

apply(
  time: Long,
  operation: String,
  operationParameters: Map[String, String],
  commandContext: Map[String, String],
  readVersion: Option[Long],
  isolationLevel: Option[String],
  isBlindAppend: Option[Boolean],
  operationMetrics: Option[Map[String, String]],
  userMetadata: Option[String]): CommitInfo

apply creates a CommitInfo (for the given arguments and based on the given commandContext for the user ID, user name, job, notebook, cluster).

commandContext argument is always empty, but could be customized using ConvertToDeltaCommandBase.

apply is used when:


Last update: 2021-03-27