Skip to content

WriteToMicroBatchDataSource Logical Operator

WriteToMicroBatchDataSource is a unary logical operator (Spark SQL) that is a top-level operator in the analyzed logical query plan of a streaming query with a sink that SupportsWrite (Spark SQL).

Creating Instance

WriteToMicroBatchDataSource takes the following to be created:

  • DataSourceV2Relation (Spark SQL) leaf logical operator
  • Table with SupportsWrite (Spark SQL)
  • Query LogicalPlan (Spark SQL)
  • Query ID
  • Write Options (Map[String, String])
  • OutputMode
  • Batch ID

WriteToMicroBatchDataSource is created when:

Query Planning

WriteToMicroBatchDataSource is optimized to WriteToDataSourceV2 logical operator by V2Writes (Spark SQL) logical optimization.