Skip to content

TaskResult

TaskResult is an abstraction of task results (of type T).

The decision what TaskResult type to use is made when TaskRunner finishes running a task.

Sealed Trait

TaskResult is a Scala sealed trait which means that all of the implementations are in the same compilation unit (a single file).

DirectTaskResult

DirectTaskResult is a TaskResult to be serialized and sent over the wire to the driver together with the following:

DirectTaskResult is used when the size of a task result is below spark.driver.maxResultSize and the maximum size of direct results.

IndirectTaskResult

IndirectTaskResult is a "pointer" to a task result that is available in a BlockManager:

IndirectTaskResult is a java.io.Serializable.

Externalizable

DirectTaskResult is an Externalizable (Java).

Back to top