Skip to content

KafkaStreamWriterFactory

KafkaStreamWriterFactory is a StreamingDataWriterFactory.

Creating Instance

KafkaStreamWriterFactory takes the following to be created:

  • Topic Name (optional)
  • KafkaProducer Parameters
  • Schema (StructType)

KafkaStreamWriterFactory is created when:

createWriter

createWriter(
  partitionId: Int,
  taskId: Long,
  epochId: Long): DataWriter[InternalRow]

createWriter is part of the StreamingDataWriterFactory abstraction.


createWriter creates a KafkaDataWriter (Spark SQL) for the given topic, KafkaProducer parameters and schema.