Skip to main content

Apache Kafka Setup

Image
By

Apache Kafka Setup

Kafka is made up of two components, I am using Kafka in Windows with minimal configuration.
  • Kafka
  • apache-zookeeper
To visualize Kakfa we will use Kafdrop, we can visualize configured topics, stats and messages from Kafka.

Setup Apache-Zookeeper

  • Download Apache Zookeeper compatible with Kafka, I am using apache-zookeeper-3.5.8-bin
  • Extract contents and make the below changes in zoo.cfg under conf folder.
            dataDir=data    (modify)
  • Start Zookeeper by running zkServer.cmd from bin folder

Setup Kafka

  • Download Kafka, I am using kafka_2.13-2.5.0
  • Extract contents and make the below changes in server.properties under conf folder.
            log.dirs=\logs\   (modify)
            delete.topic.enable=true  (add)
  • Start Kafka using the below command from Kafka folder from command line
            .\bin\windows\kafka-server-start.bat .\config\server.properties

Setup Kafkadrop

  • Setup maven environemt
  • Download Kafkadrop
  • Run the below command to create standalone package, this will create a JAR file
            mvn clean package
  • Start Kafkadrop with the below command, I have Zookeper and Kafka in same machine. Replace host and IP address to required
        java -jar ./kafdrop-2.0.0.jar --zookeeper.connect=localhost:2181 --kafka.brokers=localhost:9092

Lets create a topic and send sample message using Kafka utilities and finally we will visualize using Kafkadrop.

  • Once all the above are up and running you should see the below. 


  • Create Topic using kafka-topics.bat from bin folder using below command.
            .\kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
  • Push sample message to Kafka
            .\kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test

  • Visualize using Kafkadrop
New Topic: test
Topic Stats
Messages from topic:

Labels: