Skip to content

DataWritingCommandExec Physical Operator

DataWritingCommandExec is a UnaryExecNode that is the execution environment for a DataWritingCommand logical command at execution time.

Creating Instance

DataWritingCommandExec takes the following to be created:

DataWritingCommandExec is created when:

Performance Metrics

metrics: Map[String, SQLMetric]

metrics requests the DataWritingCommand for the metrics.

metrics is part of the SparkPlan abstraction.

sideEffectResult

sideEffectResult: Seq[InternalRow]
Lazy Value

sideEffectResult is a Scala lazy value to guarantee that the code to initialize it is executed once only (when accessed for the first time) and the computed value never changes afterwards.

Learn more in the Scala Language Specification.

sideEffectResult requests the DataWritingCommand to run (with the active SparkSession and the child logical operator) that produces output Rows.

In the end, sideEffectResult creates a Catalyst converter (for the schema) to convert the output rows.

Used when:

Executing Physical Operator

doExecute(): RDD[InternalRow]

doExecute requests the SparkPlan to parallelize the sideEffectResult (with 1 partition).

doExecute is part of the SparkPlan abstraction.