Skip to content

StreamingExecutionRelation Leaf Logical Operator

StreamingExecutionRelation is a leaf logical operator (Spark SQL) that represents a streaming source in the logical query plan of a streaming query.

The main use of StreamingExecutionRelation logical operator is to be a "placeholder" in a logical query plan that will be replaced with the real relation (with new data that has arrived since the last batch) or an empty LocalRelation when StreamExecution is requested to transforming logical plan to include the Sources and MicroBatchReaders with new data.

Note

Right after StreamExecution has started running streaming batches it initializes the streaming sources by transforming the analyzed logical plan of the streaming query so that every StreamingRelation logical operator is replaced by the corresponding StreamingExecutionRelation.

StreamingExecutionRelation Represents Streaming Source At Execution

Note

StreamingExecutionRelation is also resolved (planned) to a StreamingRelationExec physical operator in StreamingRelationStrategy execution planning strategy only when explaining a streaming Dataset.

Creating Instance

StreamingExecutionRelation takes the following to be created:

  • BaseStreamingSource
  • Output Attributes (Seq[Attribute])
  • SparkSession

StreamingExecutionRelation is created when: