Skip to content


== [[TaskResult]] TaskResults -- DirectTaskResult and IndirectTaskResult

TaskResult models a task result. It has exactly two concrete implementations:

  1. <> is the TaskResult to be serialized and sent over the wire to the driver together with the result bytes and accumulators.
  2. <> is the TaskResult that is just a pointer to a task result in a BlockManager.

The decision of the concrete TaskResult is made when a[TaskRunner finishes running a task and checks the size of the result].

NOTE: The types are private[spark].

=== [[DirectTaskResult]] DirectTaskResult Task Result

[source, scala]

DirectTaskResultT extends TaskResult[T] with Externalizable

DirectTaskResult is the <> of[running a task] (that is later[returned serialized to the driver]) when the size of the task's result is smaller than[spark.driver.maxResultSize] and[spark.task.maxDirectResultSize] (or[spark.rpc.message.maxSize] whatever is smaller).

NOTE: DirectTaskResult is Java's[].

=== [[IndirectTaskResult]] IndirectTaskResult Task Result

[source, scala]

IndirectTaskResultT extends TaskResult[T] with Serializable

IndirectTaskResult is a <> that...

NOTE: IndirectTaskResult is Java's[].

Last update: 2020-10-06