StreamingDataWriterFactory¶

StreamingDataWriterFactory is an abstraction of factories to create a DataWriter.

Contract¶

Creating DataWriter¶

DataWriter<InternalRow> createWriter(
  int partitionId,
  long taskId,
  long epochId)

Creates a DataWriter (Spark SQL)

Used when:

• ContinuousWriteRDD is requested to compute a partition
• MicroBatchWriterFactory is requested to create a DataWriter

Implementations¶

• ForeachWriterFactory
• KafkaStreamWriterFactory
• MemoryWriterFactory
• NoopStreamingDataWriterFactory
• PackedRowWriterFactory