TaskResult¶
TaskResult is an abstraction of task results (of type T).
The decision what TaskResult type to use is made when TaskRunner finishes running a task.
Sealed Trait
TaskResult is a Scala sealed trait which means that all of the implementations are in the same compilation unit (a single file).
DirectTaskResult¶
DirectTaskResult is a TaskResult to be serialized and sent over the wire to the driver together with the following:
- Value Bytes (java.nio.ByteBuffer)
- Accumulator updates
- Metric Peaks
DirectTaskResult is used when the size of a task result is below spark.driver.maxResultSize and the maximum size of direct results.
IndirectTaskResult¶
IndirectTaskResult is a "pointer" to a task result that is available in a BlockManager:
- BlockId
- Size
IndirectTaskResult is a java.io.Serializable.
Externalizable¶
DirectTaskResult is an Externalizable (Java).