OptimizeMetadataOnlyDeltaQuery¶

OptimizeMetadataOnlyDeltaQuery is an abstraction of metadata-only PrepareDeltaScans.

Contract¶

getDeltaScanGenerator¶

getDeltaScanGenerator(
  index: TahoeLogFileIndex): DeltaScanGenerator

DeltaScanGenerator for the given TahoeLogFileIndex

See:

• PrepareDeltaScanBase

Used when:

• CountStarDeltaTable is requested to extractGlobalCount

Implementations¶

• PrepareDeltaScanBase

optimizeQueryWithMetadata¶

optimizeQueryWithMetadata(
  plan: LogicalPlan): LogicalPlan

optimizeQueryWithMetadata uses stats.numRecords statistic for CountStarDeltaTable queries and so making them very fast by being metadata-only.

optimizeQueryWithMetadata transforms the given LogicalPlan (Spark SQL) (with subqueries, children and every node itself) to replace a CountStarDeltaTable with a LocalRelation (Spark SQL).

optimizeQueryWithMetadata is used when:

• PrepareDeltaScanBase logical optimization is requested to execute

extractGlobalCount¶

extractGlobalCount(
  tahoeLogFileIndex: TahoeLogFileIndex): Option[Long]

extractGlobalCount...FIXME

extractGlobalCount is used when:

• CountStarDeltaTable is requested to destruct Aggregate logical operator
• ShowCountStarDeltaTable is requested to destruct Aggregate logical operator