DataWriterFactory¶

DataWriterFactory is an abstraction of DataWriter factories.

DataWriterFactory is Serializable.

Contract¶

Creating DataWriter¶

DataWriter<InternalRow> createWriter(
  int partitionId,
  long taskId)

Creates a DataWriter (for the given partitionId and taskId)

Used when:

• DataWritingSparkTask is requested to run

Implementations¶

• FileWriterFactory
• KafkaBatchWriterFactory
• MemoryWriterFactory (Spark Structured Streaming)
• MicroBatchWriterFactory (Spark Structured Streaming)
• NoopWriterFactory