Consuming Kafka Messages with Confluent REST Proxy

30 minutes
  • 2 Learning Objectives

About this Hands-on Lab

The Confluent REST Proxy provides a “RESTful interface” on top of Kafka, allowing you to produce and consume messages using simple HTTP requests. In this lab, you will have the opportunity to interact with the REST proxy by consuming some existing messages. This will give you some hands-on experience with the requests necessary for consuming Kafka data using REST.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Provide the Number of Apples Sold in the Last Week
  1. Create a new consumer and consumer_instance:

    curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" 
    --data '{"name": "sales_consumer_instance", "format": "json", "auto.offset.reset": "earliest"}' 
    http://localhost:8082/consumers/sales_json_consumer
  2. Subscribe the consumer to the weekly_sales topic:

    curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data '{"topics":["weekly_sales"]}' 
    http://localhost:8082/consumers/sales_json_consumer/instances/sales_consumer_instance/subscription
  3. Consume the messages:

    curl -X GET -H "Accept: application/vnd.kafka.json.v2+json" 
    http://localhost:8082/consumers/sales_json_consumer/instances/sales_consumer_instance/records
  4. Locate the latest record with a key of apples and copy its value.

  5. Edit the output file:

    vi /home/cloud_user/output/apple_sales.txt
  6. Paste in the value from the latest apples record and save the file.

Provide the Current Quarterly Balance for Product Purchases
  1. Create a new consumer and consumer_instance:

    curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" 
    --data '{"name": "balance_consumer_instance", "format": "json", "auto.offset.reset": "earliest"}' 
    http://localhost:8082/consumers/balance_json_consumer
  2. Subscribe the consumer to the quarterly_purchases topic:

    curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data '{"topics":["quarterly_purchases"]}' 
    http://localhost:8082/consumers/balance_json_consumer/instances/balance_consumer_instance/subscription
  3. Consume the messages:

    curl -X GET -H "Accept: application/vnd.kafka.json.v2+json" 
    http://localhost:8082/consumers/balance_json_consumer/instances/balance_consumer_instance/records
  4. Locate the latest record and copy its value.

  5. Edit the output file:

    vi /home/cloud_user/output/quarterly_balance.txt
  6. Paste in the value from the latest record and save the file.

Additional Resources

Your supermarket company is using Kafka to handle messaging as part of its infrastructure. They want to prepare a report that requires some data that is currently stored in Kafka.

You have been asked to access the cluster and provide some data points that will be used in this report. Luckily, the Confluent REST Proxy will make it easy for you to gather the necessary data using simple HTTP requests. Obtain the requested data points and place them in the specified output files.

First, the report will need to include the number of apples sold in the last week. This information can be found in a topic called weekly_sales. The records in this topic represent aggregate data. Find the latest record with a key of apples and write its value to the file located at /home/cloud_user/output/apple_sales.txt.

Secondly, the report needs to include the current quarterly balance for product purchases. Read from the topic called quarterly_purchases. Find the latest record and write its value to the file located at /home/cloud_user/output/quarterly_balance.txt.

If you get stuck, feel free to check out the solution video, or the detailed instructions under each objective. Good luck!

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?