Running Kafka within Docker

Note: This post was written a while back but sat in draft. I’ve published this now, but I’m not sure it’s relevant to the latest versions etc. so please bear this in mind.

Kafka

Kafka is an event streaming service.

Let’s pull the image

docker pull confluentinc/cp-kafka

The docker image of Kakfa requires Zookeeper.

Putting it all together

Create yourself a docker-compose.yml (this one below is taken from Guide to Setting Up Apache.

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - 22181:2181
  
  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

From your server, you can now connect to Kafka using the docker name (hence replace the docker-name with the name you gave or docker gave to your Kafka instance)

docker exec -it docker-name /bin/sh