Skip to content

TaskResult

TaskResult is an abstraction of task results (of type T).

The decision what TaskResult type to use is made when TaskRunner finishes running a task.

Sealed Trait

TaskResult is a Scala sealed trait which means that all of the implementations are in the same compilation unit (a single file).

sealed trait TaskResult[T]

DirectTaskResult

DirectTaskResult is a TaskResult to be serialized and sent over the wire to the driver together with the following:

DirectTaskResult is used when the size of a task result is below spark.driver.maxResultSize and the maximum size of direct results.

DirectTaskResult is a java.io.Externalizable.

IndirectTaskResult

IndirectTaskResult is a "pointer" to a task result that is available in a BlockManager:

IndirectTaskResult is a java.io.Serializable.


Last update: 2020-12-02