Auto Compaction¶

Auto Compaction feature allows performing OPTIMIZE command at the end of a transaction (and compacting files upon a successful write into a delta table).

Auto Compaction can be enabled system-wide or per table using spark.databricks.delta.autoCompact.enabled configuration property or delta.autoOptimize.autoCompact table property, respectively.

Auto Compaction uses AutoCompact post-commit hook to be executed at a successful transaction commit if there are files written to a delta table and it even makes sense to run such a heavy file rewritting job.

Eventually, Auto Compaction uses OptimizeExecutor (with no zOrderByColumns and the isAutoCompact flag enabled) to run optimization.

Auto Compaction uses the following configuration properties:

• spark.databricks.delta.autoCompact.maxFileSize
• spark.databricks.delta.autoCompact.minFileSize