Kafka Data Source Options¶
includeHeaders¶
default: false
[[options]] .Kafka Data Source Options [cols="1m,1,2",options="header",width="100%"] |=== | Option | Default | Description
| assign |
|---|
| [[assign]] One of the three subscription strategy options (with subscribe and subscribepattern) |
See KafkaSourceProvider.strategy
| endingoffsets |
|---|
| [[endingoffsets]] |
| failondataloss |
|---|
| [[failondataloss]] |
| kafkaConsumer.pollTimeoutMs |
|---|
| [[kafkaConsumer.pollTimeoutMs]] See kafkaConsumer.pollTimeoutMs |
| startingoffsets |
|---|
| [[startingoffsets]] |
| subscribe |
|---|
| [[subscribe]] One of the three subscription strategy options (with subscribepattern and assign) |
See KafkaSourceProvider.strategy
| subscribepattern |
|---|
| [[subscribepattern]] One of the three subscription strategy options (with subscribe and assign) |
See KafkaSourceProvider.strategy
| topic | a| [[topic]] Required for writing a DataFrame to Kafka
Used when:
-
KafkaSourceProvideris requested to write a DataFrame to a Kafka topic and create a BaseRelation afterwards -
(Spark Structured Streaming)
KafkaSourceProvideris requested tocreateStreamWriterandcreateSink|===