Skip to content

QueryStageExec Leaf Physical Operators

QueryStageExec is an extension of the LeafExecNode abstraction for leaf physical operators for Adaptive Query Execution.

Contract

Cancelling

cancel(): Unit

Cancels the stage materialization if in progress; otherwise does nothing.

Used when:

Materializing

doMaterialize(): Future[Any]

Used when:

Runtime Statistics

getRuntimeStatistics: Statistics

Statistics after stage materialization

Used when:

Unique ID

id: Int

Used when:

newReuseInstance

newReuseInstance(
  newStageId: Int,
  newOutput: Seq[Attribute]): QueryStageExec

Used when:

Physical Query Plan

plan: SparkPlan

The sub-tree of the main query plan of this query stage (that acts like a child operator, but QueryStageExec is a LeafExecNode and has no children)

Implementations

Result

_resultOption: AtomicReference[Option[Any]]

QueryStageExec uses a _resultOption transient volatile internal variable (of type AtomicReference) for the result of a successful materialization of the QueryStageExec operator (when preparing for query execution):

As AtomicReference is mutable that is enough to change the value.

_resultOption is set when AdaptiveSparkPlanExec physical operator is requested for the final physical plan.

_resultOption is available using resultOption.

resultOption

resultOption: AtomicReference[Option[Any]]

resultOption returns the current value of the _resultOption registry.

resultOption is used when:

Statistics

computeStats(): Option[Statistics]

If this QueryStageExec has been materialized, computeStats gives a new Statistics with the runtime statistics. Otherwise, computeStats returns no statistics (None).

computeStats is used when:

  • LogicalQueryStage logical operator is requested for the Statistics

Materializing Query Stage

materialize(): Future[Any]

materialize prints out the following DEBUG message to the logs (with the id):

Materialize query stage [simpleName]: [id]

materialize doMaterialize.

Final Method

materialize is a Scala final method and may not be overridden in subclasses.

Learn more in the Scala Language Specification.

materialize is used when:

Text Representation

generateTreeString(
  depth: Int,
  lastChildren: Seq[Boolean],
  append: String => Unit,
  verbose: Boolean,
  prefix: String = "",
  addSuffix: Boolean = false,
  maxFields: Int,
  printNodeId: Boolean,
  indent: Int = 0): Unit

generateTreeString is part of the TreeNode abstraction.

generateTreeString generateTreeString (the default) followed by another generateTreeString (with the depth incremented).