Kafka Data Source Options¶
includeHeaders¶
default: false
[[options]] .Kafka Data Source Options [cols="1m,1,2",options="header",width="100%"] |=== | Option | Default | Description
assign |
---|
[[assign]] One of the three subscription strategy options (with subscribe and subscribepattern) |
See KafkaSourceProvider.strategy
endingoffsets |
---|
[[endingoffsets]] |
failondataloss |
---|
[[failondataloss]] |
kafkaConsumer.pollTimeoutMs |
---|
[[kafkaConsumer.pollTimeoutMs]] See kafkaConsumer.pollTimeoutMs |
startingoffsets |
---|
[[startingoffsets]] |
subscribe |
---|
[[subscribe]] One of the three subscription strategy options (with subscribepattern and assign) |
See KafkaSourceProvider.strategy
subscribepattern |
---|
[[subscribepattern]] One of the three subscription strategy options (with subscribe and assign) |
See KafkaSourceProvider.strategy
| topic | a| [[topic]] Required for writing a DataFrame to Kafka
Used when:
-
KafkaSourceProvider
is requested to write a DataFrame to a Kafka topic and create a BaseRelation afterwards -
(Spark Structured Streaming)
KafkaSourceProvider
is requested tocreateStreamWriter
andcreateSink
|===