CollectMetricsExec Physical Operator¶
CollectMetricsExec is a unary physical operator.
CollectMetricsExec takes the following to be created:
Collected metrics Accumulator¶
CollectMetricsExec registers an AggregatingAccumulator under the name Collected metrics.
Executing Physical Operator¶
doExecute is part of the SparkPlan abstraction.
doExecute resets the Collected metrics Accumulator.
- A new per-partition AggregatingAccumulator (called
updater) is requested to copyAndReset
- The value of the accumulator is published only when a task is completed
- For every row, the per-partition
AggregatingAccumulatoris requested to add it (that updates ImperativeAggregates and TypedImperativeAggregates)