Using Client Authentication with Kafka

30 minutes
  • 3 Learning Objectives

About this Hands-on Lab

If you want to expose your Kafka brokers so that external clients can communicate with them, it is important to make sure that your implementation is secure. In some situations, this means implementing client authentication. In this lab, you will have the opportunity to work hands-on with one of the client authentication methods available in Kafka: client certificates. You will implement authentication with client certificates in an existing cluster, and then authenticate as a client to verify that your implementation works. This will give you hands-on experience with the process of securing a Kafka cluster.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Generate Your Client Certificate Files
  1. Generate a client certificate. Choose a password for the client keystore when prompted:

    cd ~/certs/
    
    keytool -keystore client.keystore.jks -alias kafkauser -validity 365 -genkey -keyalg RSA -dname "CN=kafkauser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown"
  2. Sign the key, then import the certificate authority and signed key into the keystore. When asked for the password to the ca-key, enter the password AllTheKeys:

    keytool -keystore client.keystore.jks -alias kafkauser -certreq -file client-cert-file
    
    openssl x509 -req -CA ca-cert -CAkey ca-key -in client-cert-file -out client-cert-signed -days 365 -CAcreateserial
    
    keytool -keystore client.keystore.jks -alias CARoot -import -file ca-cert
    
    keytool -keystore client.keystore.jks -alias kafkauser -import -file client-cert-signed
  3. Move the client keystore into an appropriate location:

    sudo cp client.keystore.jks /var/private/ssl/
    
    sudo chown root:root /var/private/ssl/client.keystore.jks
Enable Client Authentication for the Broker
  1. Set client authentication to required in server.properties:

    sudo vi /etc/kafka/server.properties
  2. Locate the line that begins with ssl.client.auth and change it:

    ssl.client.auth=required
  3. Restart Kafka and then verify that everything is working:

    sudo systemctl restart confluent-kafka
    
    sudo systemctl status confluent-kafka
Add Client Authentication Settings to Your Client Config File
  1. Edit client-ssl.properties:

    cd ~/
    
    vi client-ssl.properties
  2. Add the following lines:

    ssl.keystore.location=/var/private/ssl/client.keystore.jks
    ssl.keystore.password=<your client keystore password>
    ssl.key.password=<your client key password>
  3. Create a console consumer using client authentication to make verify that everything is working:

    kafka-console-consumer --bootstrap-server zoo1:9093 --topic inventory_purchases --from-beginning --consumer.config client-ssl.properties

Additional Resources

Your supermarket company is using Kafka as part of its backend data infrastructure. The cluster has already been secured with TLS on a secure port, but currently, any client using that secure port has full access to everything in the cluster.

You have been asked to implement client authentication using client certificates. The purpose of this task is so that only clients with client certificates signed by the cluster's certificate authority can use the secure port.

To complete this task, you will need the following information:

  • The certificate authority files you need in order to sign the client certificate (ca-cert and ca-key) can be found in /home/cloud_user/certs.
  • The password for the ca-key is AllTheKeys.
  • There is a client configuration file located at /home/cloud_user/client-ssl.properties.
  • There is a topic called inventory_purchases with a few test records. You can consume from this topic in order to test your configuration like so:

    kafka-console-consumer --bootstrap-server zoo1:9093 --topic inventory_purchases --from-beginning --consumer.config client-ssl.properties

If you get stuck, feel free to check out the solution video, or the detailed instructions under each objective. Good luck!

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?