Kafka Connector¶
Kafka Connector allows Spark SQL (and Spark Structured Streaming) to read data from and write data to topics in Apache Kafka.
Kafka Connector is available as kafka format alias.
The entry point is KafkaSourceProvider.
Note
Apache Kafka is a storage of records in a format-independent and fault-tolerant durable way.
Learn more about Apache Kafka in the official documentation or The Internals of Apache Kafka.
Kafka Connector supports options to fine-tune structured queries.