DataWritingCommandExec Physical Operator¶
DataWritingCommandExec is a UnaryExecNode that is the execution environment for a DataWritingCommand logical command at execution time.
DataWritingCommandExec takes the following to be created:
- Child SparkPlan
DataWritingCommandExec is created when:
- BasicOperators execution planning strategy is requested to plan a DataWritingCommand logical command
metrics: Map[String, SQLMetric]
metrics requests the DataWritingCommand for the metrics.
metrics is part of the SparkPlan abstraction.
sideEffectResult is a Scala lazy value to guarantee that the code to initialize it is executed once only (when accessed for the first time) and the computed value never changes afterwards.
Learn more in the Scala Language Specification.
sideEffectResult requests the DataWritingCommand to run (with the active SparkSession and the child logical operator) that produces output Rows.
In the end,
sideEffectResult creates a Catalyst converter (for the schema) to convert the output rows.
DataWritingCommandExecis requested to executeCollect, executeToIterator, executeTake, executeTail and doExecute
Executing Physical Operator¶
doExecute requests the SparkPlan to
parallelize the sideEffectResult (with
doExecute is part of the SparkPlan abstraction.