WriteToDataSourceV2 Logical Operator¶
WriteToDataSourceV2 is a unary logical operator (Spark SQL) that represents WriteToMicroBatchDataSource unary logical operator at logical optimization time for writing data in Spark Structured Streaming.
WriteToDataSourceV2 is deprecated since Spark SQL 2.4.0 (in favour of
AppendData logical operator and alike).
WriteToDataSourceV2 takes the following to be created:
- Logical Query Plan (Spark SQL)
CustomMetrics (Spark SQL)
WriteToDataSourceV2 is created when:
V2Writes(Spark SQL) logical optimization is requested to optimize a logical query (with a WriteToMicroBatchDataSource unary logical operator)