AccumulableInfo¶

AccumulableInfo represents an update to an AccumulatorV2.

AccumulableInfo is used to transfer accumulator updates from executors to the driver every executor heartbeat or when a task finishes.

Creating Instance¶

AccumulableInfo takes the following to be created:

• Accumulator ID
• Name
• Partial Update
• Partial Value
• internal flag
• countFailedValues flag
• Metadata (default: None)

AccumulableInfo is created when:

• AccumulatorV2 is requested to convert itself to an AccumulableInfo
• JsonProtocol is requested to accumulableInfoFromJson
• SQLMetric (Spark SQL) is requested to convert itself to an AccumulableInfo

internal Flag¶

internal: Boolean

AccumulableInfo is given an internal flag when created.

internal flag denotes whether this accumulator is internal.

internal is used when:

• LiveEntityHelpers is requested for newAccumulatorInfos
• JsonProtocol is requested to accumulableInfoToJson