Skip to content

ConsumerStrategy — Kafka Consumer Providers

ConsumerStrategy is the <> for <> that can <> given Kafka parameters.

[[contract]] [source, scala]


package org.apache.spark.sql.kafka010

sealed trait ConsumerStrategy { def createConsumer(kafkaParams: ju.Map[String, Object]): Consumer[Array[Byte], Array[Byte]] }


.ConsumerStrategy Contract [cols="1m,2",options="header",width="100%"] |=== | Property | Description

| createConsumer | [[createConsumer]] Creates a Kafka https://kafka.apache.org/20/javadoc/org/apache/kafka/clients/consumer/Consumer.html[Consumer] (of keys and values of type Array[Byte])

Used exclusively when KafkaOffsetReader is requested to creating a Kafka Consumer |===

[[implementations]] .ConsumerStrategies [cols="1,2",options="header",width="100%"] |=== | ConsumerStrategy | createConsumer

| AssignStrategy | [[AssignStrategy]] Uses ++http://kafka.apache.org/20/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#assign-java.util.Collection-++[KafkaConsumer.assign(Collection partitions)]

| SubscribeStrategy | [[SubscribeStrategy]] Uses ++http://kafka.apache.org/20/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#subscribe-java.util.Collection-++[KafkaConsumer.subscribe(Collection topics)]

| SubscribePatternStrategy a| [[SubscribePatternStrategy]] Uses ++http://kafka.apache.org/20/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#subscribe-java.util.regex.Pattern-org.apache.kafka.clients.consumer.ConsumerRebalanceListener-++[KafkaConsumer.subscribe(Pattern pattern, ConsumerRebalanceListener listener)] with NoOpConsumerRebalanceListener.

TIP: Refer to http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html[java.util.regex.Pattern] for the format of supported topic subscription regex patterns. |===

NOTE: ConsumerStrategy is a Scala sealed trait which means that all the <> are in the same compilation unit (a single file).