KafkaStreamWriterFactory¶

KafkaStreamWriterFactory is a StreamingDataWriterFactory.

Creating Instance¶

KafkaStreamWriterFactory takes the following to be created:

• Topic Name (optional)
• KafkaProducer Parameters
• Schema (StructType)

KafkaStreamWriterFactory is created when:

• KafkaStreamingWrite is requested for a StreamingDataWriterFactory

createWriter¶

createWriter(
  partitionId: Int,
  taskId: Long,
  epochId: Long): DataWriter[InternalRow]

createWriter is part of the StreamingDataWriterFactory abstraction.

createWriter creates a KafkaDataWriter (Spark SQL) for the given topic, KafkaProducer parameters and schema.