MicroBatchWriterFactory¶

MicroBatchWriterFactory is a DataWriterFactory (Spark SQL).

Creating Instance¶

MicroBatchWriterFactory takes the following to be created:

• Epoch ID
• StreamingDataWriterFactory

MicroBatchWriterFactory is created when:

• MicroBatchWrite is requested to create a DataWriterFactory for batch write

Creating DataWriter¶

createWriter(
  partitionId: Int,
  taskId: Long): DataWriter[InternalRow]

createWriter is part of the DataWriterFactory (Spark SQL) abstraction.

createWriter requests the StreamingDataWriterFactory for a DataWriter.