Skip to content

Demo: Kafka and kcat in Docker

This demo uses Docker to run Apache Kafka and kcat utility.

kafka-docker

Pull kafka-docker project (or create a docker-compose.yml file yourself).

Running Kafka Cluster

Start Zookeeper and Kafka containers.

docker-compose up
$ docker-compose ps
          Name                        Command               State                                  Ports
----------------------------------------------------------------------------------------------------------------------------------------
kafka-docker_kafka_1       start-kafka.sh                   Up      0.0.0.0:62687->9092/tcp
kafka-docker_zookeeper_1   /bin/sh -c /usr/sbin/sshd  ...   Up      0.0.0.0:2181->2181/tcp,:::2181->2181/tcp, 22/tcp, 2888/tcp, 3888/tcp

Docker Network

The above creates a Docker network kafka-docker_default (if ran from kafka-docker directory as described in the official documentation of docker-compose).

$ docker network ls
NETWORK ID     NAME                   DRIVER    SCOPE
b8b255710858   bridge                 bridge    local
3c9c3a969ef2   cda                    bridge    local
398f9f3196aa   host                   host      local
68611503fde8   kafka-docker_default   bridge    local
db43a5e50281   none                   null      local

kcat

Connect kcat container to the network (using --network option as described in the official documentation of docker-compose).

Metadata Listing

docker run -it --rm \
  --network kafka-docker_default \
  edenhill/kcat:1.7.0 \
  -b kafka-docker_kafka_1:9092 -L
Metadata for all topics (from broker -1: kafka-docker_kafka_1:9092/bootstrap):
 1 brokers:
  broker 1001 at 09cc8de4d067:9092 (controller)
 0 topics:

Producer

docker run -it --rm \
  --network kafka-docker_default \
  --name producer \
  edenhill/kcat:1.7.0 \
  -b kafka-docker_kafka_1:9092 -P -t t1

Caution

For some reason the above command couldn't send messages whenever I pressed ENTER but expected Ctrl+D instead (that terminates the shell and the container). Switching to confluentinc/cp-kafkacat made things working fine.

docker run -it --rm \
  --network kafka-docker_default \
  --name producer \
  confluentinc/cp-kafkacat \
  kafkacat \
  -b kafka-docker_kafka_1:9092 -P -t t1

Consumer

docker run -it --rm \
  --network kafka-docker_default \
  --name consumer \
  edenhill/kcat:1.7.0 \
  -b kafka-docker_kafka_1:9092 -C -t t1

Clean Up

docker-compose down
Back to top