Practitioner Level

Connecting to Kafka Programmatically in Java

Pricing
1.25 hours
  • 2 Learning Objectives

About this Hands-on Lab

The real power of Kafka comes from the ability to write applications that utilize it. Kafka’s Java APIs make it easy to build applications that work with Kafka data. Through this lab, we will have the opportunity to build a simple Java program that consumes data from a Kafka topic. Tackling this challenge will provide hands-on experience with the Kafka Java APIs and consuming data from a real cluster. After completing this introductory lab, we will be ready to move on to more complex scenarios involving both Kafka and Java code.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Clone the Starter Project from GitHub and Perform a Test Run

  1. Clone the starter project from GitHub.

    cd ~/
git clone https://github.com/linuxacademy/content-ccdak-kafka-simple-consumer.git

  2. Perform a test to make sure the code is able to compile and run.

    cd content-ccdak-kafka-simple-consumer/
./gradlew run

    The output should contain the message printed by the Main class: Hello, world!.

Implement a Consumer in the Main Class and Run It

  1. Add the Kafka Client Libraries as a project dependency in build.gradle with:

    vi build.gradle

  2. In the dependencies { ... } block, add the following line:

    implementation 'org.apache.kafka:kafka-clients:2.2.1'

  3. Edit the Main class:

    vi src/main/java/com/linuxacademy/ccdak/kafkaSimpleConsumer/Main.java

  4. Implement a basic consumer that consumes messages from the topic and prints them to the screen:

    package com.linuxacademy.ccdak.kafkaSimpleConsumer;

import org.apache.kafka.clients.consumer.*;
import java.util.Properties;
import java.util.Arrays;
import java.time.Duration;

public class Main {

  public static void main(String[] args) {
      Properties props = new Properties();
      props.setProperty("bootstrap.servers", "localhost:9092");
      props.setProperty("group.id", "my-group");
      props.setProperty("enable.auto.commit", "true");
      props.setProperty("auto.commit.interval.ms", "1000");
      props.setProperty("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
      props.setProperty("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
      KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(props);
      consumer.subscribe(Arrays.asList("inventory_purchases"));
      while (true) {
          ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
          for (ConsumerRecord<String, String> record : records) {
              System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());
          }
      }
  }

}

  5. Run the code:

    ./gradlew run

    The program should print a series of messages from the Kafka topic containing information about item purchases.

Contact SalesSee Pricing

Additional Resources

Your supermarket company is using Kafka to track and process purchase data to dynamically keep track of inventory changes. They have a topic called inventory purchases, data about the type, plus the number of items purchased that get published to this topic.

You have been asked to begin the process of writing a Java application that can consume this data. As an initial step, your current task is to write a simple Java program that can consume messages from the Kafka topic and print it to the console.

You can find a starter java project here: https://github.com/linuxacademy/content-ccdak-kafka-simple-consumer Clone the starter project into your home folder and implement the Kafka consumer inside the Main class.

This project includes a basic project framework and a Gradle build to allow you to compile run the code easily. From within the leading project directory, you can run the code in the Main class with the command ./gradlew run.

You can find some examples of Kafka consumer code in the Kafka Consumer API Javadocs. The GitHub starter project also includes an example solution in the end-state branch.

Do all of your work on the Broker 1 server. You can access the Kafka cluster from that server at localhost:9092.

If you get stuck, feel free to check out the solution video, or the detailed instructions under each objective. Good luck!

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Don't have an account?

Get Started
Who’s going to be learning?
MyselfMy Organization