Configuring Kafka producers is a relatively simple process. The real challenge is knowing what configuration options to change in order to tune the producer for your use case. In this lab, you will have the opportunity to work hands-on with the process of tuning a Kafka producer. You will be presented with a few issues that can be addressed through configuration tuning. You will then need to determine what configuration changes to make in order to address these issues, and you will need to implement those changes in the producer.
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Clone the Starter Project from GitHub and Perform a Test Run
Clone the starter project from GitHub.:
cd ~/ git clone https://github.com/linuxacademy/content-ccdak-producer-tuning-lab.gitPerform a test run to make sure the code is able to compile and run:
cd content-ccdak-producer-tuning-lab ./gradlew runThe code should compile, and tests should succeed.
- Implement the Configuration Changes in the Producer
Open the file:
vi src/main/java/com/linuxacademy/ccdak/producer/MemberSignupsProducer.javaFix the issue with data loss by setting
acks=all:props.put("acks", "all");Fix the out-of-order record issue caused by retries by setting
max.in.flight.requests.per.connection=1:props.put("max.in.flight.requests.per.connection", "1");Increase the batch size by setting
batch.size=65536:props.put("batch.size", "65536");Run your code to verify that it still works:
./gradlew run
