SQLMetricsReporting¶

SQLMetricsReporting is an extension for OptimisticTransactionImpl to track performance metrics of Operations for reporting.

Implementations¶

• OptimisticTransactionImpl

operationSQLMetrics Registry¶

operationSQLMetrics: Map[String, SQLMetric]

SQLMetricsReporting uses operationSQLMetrics internal registry for SQLMetrics (Spark SQL) by their names.

SQLMetrics are registered only when spark.databricks.delta.history.metricsEnabled configuration property is enabled.

operationSQLMetrics is used when SQLMetricsReporting is requested for the following:

• Operation Metrics
• getMetric

Registering SQLMetrics¶

registerSQLMetrics(
  spark: SparkSession,
  metrics: Map[String, SQLMetric]): Unit

registerSQLMetrics adds (registers) the given metrics to the operationSQLMetrics internal registry only when spark.databricks.delta.history.metricsEnabled configuration property is enabled.

registerSQLMetrics is used when:

• DeleteCommand, MergeIntoCommand, UpdateCommand commands are executed
• TransactionalWrite is requested to writeFiles
• DeltaSink is requested to addBatch

Operation Metrics¶

getMetricsForOperation(
  operation: Operation): Map[String, String]

getMetricsForOperation requests the given Operation to transform the operation metrics.

getMetricsForOperation is used when:

• OptimisticTransactionImpl is requested for the operation metrics

Looking Up Operation Metric¶

getMetric(
  name: String): Option[SQLMetric]

getMetric uses the operationSQLMetrics registry to look up the SQLMetric by name.

getMetric is used when:

• UpdateCommand is executed