Skip to content

Kafka Data Source

Kafka Data Source allows Spark SQL (and Spark Structured Streaming) to read data from and write data to topics in Apache Kafka.

Kafka Data Source is available as kafka format alias.

The entry point is KafkaSourceProvider.

Note

Apache Kafka is a storage of records in a format-independent and fault-tolerant durable way.

Learn more about Apache Kafka in the official documentation or The Internals of Apache Kafka.

Kafka Data Source supports options to fine-tune structured queries.