Encrypt Cluster and Client Elasticsearch Networks

1.75 hours
  • 5 Learning Objectives

About this Hands-on Lab

No matter what technology we are working with, we always need to be mindful of security. Big data platforms are certainly no exception, as they can contain massive amounts of sensitive data that must be protected. Elasticsearch provides multiple security mechanisms through X-Pack Security. In this hands-on lab, you will complete the following tasks on a 3-node Elasticsearch cluster:

* Generate a certificate authority
* Generate node certificates
* Add certificate passwords to Elasticsearch’s keystore
* Enable cluster (transport) network encryption
* Enable client (HTTP) network encryption
* Set built-in user passwords

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Generate a Certificate Authority (CA).
  1. Using the Secure Shell (SSH), log in to each node as cloud_user via the public IP address.

  2. Become the elastic user with:

sudo su - elastic
  1. Create a certs directory on each node:
mkdir /home/elastic/elasticsearch/config/certs
  1. On the master-1 node, create a CA certificate with password elastic_ca in the new certs directory:
/home/elastic/elasticsearch/bin/elasticsearch-certutil ca --out config/certs/ca --pass elastic_ca
Generate and deploy a certificate for each node per instructions.
  1. Using the Secure Shell (SSH), log in to the master-1 node as cloud_user via the public IP address.

  2. Become the elastic user with:

sudo su - elastic
  1. On the master-1 node, generate each node’s certificate with the CA per instructions:
/home/elastic/elasticsearch/bin/elasticsearch-certutil cert --ca config/certs/ca --ca-pass elastic_ca --name master-1 --dns ip-10-1-101.ec2.internal --ip 10.0.1.101 --out config/certs/master-1 --pass elastic_master_1
/home/elastic/elasticsearch/bin/elasticsearch-certutil cert --ca config/certs/ca --ca-pass elastic_ca --name master-2 --dns ip-10-1-102.ec2.internal --ip 10.0.1.102 --out config/certs/master-2 --pass elastic_master_2
/home/elastic/elasticsearch/bin/elasticsearch-certutil cert --ca config/certs/ca --ca-pass elastic_ca --name master-3 --dns ip-10-1-103.ec2.internal --ip 10.0.1.103 --out config/certs/master-3 --pass elastic_master_3
/home/elastic/elasticsearch/bin/elasticsearch-certutil cert --ca config/certs/ca --ca-pass elastic_ca --name data-1 --dns ip-10-1-104.ec2.internal --ip 10.0.1.104 --out config/certs/data-1 --pass elastic_data_1
/home/elastic/elasticsearch/bin/elasticsearch-certutil cert --ca config/certs/ca --ca-pass elastic_ca --name data-2 --dns ip-10-1-105.ec2.internal --ip 10.0.1.105 --out config/certs/data-2 --pass elastic_data_2
/home/elastic/elasticsearch/bin/elasticsearch-certutil cert --ca config/certs/ca --ca-pass elastic_ca --name data-3 --dns ip-10-1-106.ec2.internal --ip 10.0.1.106 --out config/certs/data-3 --pass elastic_data_3
  1. On the master-1 node, remote copy each certificate to the certs directory created on each node:
scp /home/elastic/elasticsearch/config/certs/master-2 10.0.1.102:/home/elastic/elasticsearch/config/certs
scp /home/elastic/elasticsearch/config/certs/master-3 10.0.1.103:/home/elastic/elasticsearch/config/certs
scp /home/elastic/elasticsearch/config/certs/data-1 10.0.1.104:/home/elastic/elasticsearch/config/certs
scp /home/elastic/elasticsearch/config/certs/data-2 10.0.1.105:/home/elastic/elasticsearch/config/certs
scp /home/elastic/elasticsearch/config/certs/data-3 10.0.1.106:/home/elastic/elasticsearch/config/certs
  1. Add the transport keystore password on each node:
echo "CERTIFICATE_PASSWORD_HERE" | /home/elastic/elasticsearch/bin/elasticsearch-keystore add --stdin xpack.security.transport.ssl.keystore.secure_password
  1. Add the transport truststore password on each node:
echo "CERTIFICATE_PASSWORD_HERE" | /home/elastic/elasticsearch/bin/elasticsearch-keystore add --stdin xpack.security.transport.ssl.truststore.secure_password
  1. Add the HTTP keystore password on each node:
echo "CERTIFICATE_PASSWORD_HERE" | /home/elastic/elasticsearch/bin/elasticsearch-keystore add --stdin xpack.security.http.ssl.keystore.secure_password
  1. Add the HTTP truststore password on each node:
echo "CERTIFICATE_PASSWORD_HERE" | /home/elastic/elasticsearch/bin/elasticsearch-keystore add --stdin xpack.security.http.ssl.truststore.secure_password
Configure transport network encryption and restart Elasticsearch.
  1. Using the Secure Shell (SSH), log in to each node as cloud_user via the public IP address.

  2. Become the elastic user with:

sudo su - elastic
  1. Add the following to /home/elastic/elasticsearch/config/elasticsearch.yml on each node:
#
# ---------------------------------- X-Pack ------------------------------------
#
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.transport.ssl.keystore.path: certs/CERTIFICATE_FILE_NAME_HERE
xpack.security.transport.ssl.truststore.path: certs/CERTIFICATE_FILE_NAME_HERE
  1. Stop Elasticsearch:
pkill -F /home/elastic/elasticsearch/pid
  1. Start Elasticsearch as a background daemon and record the PID to a file:
/home/elastic/elasticsearch/bin/elasticsearch -d -p pid
Use the elasticsearch-setup-passwords tool to set the password for each built-in user.
  1. Using the Secure Shell (SSH), log in to the master-1 node as cloud_user via the public IP address.

  2. Become the elastic user with:

sudo su - elastic
  1. Set the built-in user passwords using the elasticsearch-setup-passwords utility on the master-1 node:
/home/elastic/elasticsearch/bin/elasticsearch-setup-passwords interactive
  1. Use the following passwords:
User: elastic
Password: la_elastic_409

User: apm_system
Password: la_apm_system_409

User: kibana
Password: la_kibana_409

User: logstash_system
Password: la_logstash_system_409

User: beats_system
Password: la_beats_system_409

User: remote_monitoring_user
Password: la_remote_monitoring_user_409
Configure HTTP network encryption and restart Elasticsearch.
  1. Using the Secure Shell (SSH), log in to each node as cloud_user via the public IP address.

  2. Become the elastic user with:

sudo su - elastic
  1. Add the following to /home/elastic/elasticsearch/config/elasticsearch.yml:
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/CERTIFICATE_FILE_NAME_HERE
xpack.security.http.ssl.truststore.path: certs/CERTIFICATE_FILE_NAME_HERE
  1. Stop Elasticsearch:
pkill -F /home/elastic/elasticsearch/pid
  1. Start Elasticsearch as a background daemon and record the PID to a file:
/home/elastic/elasticsearch/bin/elasticsearch -d -p pid

Additional Resources

You work as a system administrator and are in charge of a 6-node Elasticsearch cluster that has been used for ad-hoc, non-sensitive data analysis. However, your Business Intelligence office would like to start using this cluster to analyze some sensitive customer data, so they need you to secure the Elasticsearch cluster to meet the following data security requirements:

  • Cluster network encryption
  • Client network encryption
  • User authentication

To do this, you will need to create a certificate authority (CA) protected by the password elastic_ca using the elasticsearch-certutil tool, and then use the CA to create and sign node certificates for each Elasticsearch node. The node certificates should also be password-protected and stored in the /home/elastic/elasticsearch/config/certs directory.

To support full verification mode, the node certificates should be generated as follows:

+----------+--------------------------+------------+----------+------------------+
| Name     | DNS                      | IP         | Filename | Password         |
+----------+--------------------------+------------+----------+------------------+
| master-1 | ip-10-1-101.ec2.internal | 10.0.1.101 | master-1 | elastic_master_1 |
+----------+--------------------------+------------+----------+------------------+
| master-2 | ip-10-1-102.ec2.internal | 10.0.1.102 | master-2 | elastic_master_2 |
+----------+--------------------------+------------+----------+------------------+
| master-3 | ip-10-1-103.ec2.internal | 10.0.1.103 | master-3 | elastic_master_3 |
+----------+--------------------------+------------+----------+------------------+
| data-1   | ip-10-1-104.ec2.internal | 10.0.1.104 | data-1   | elastic_data_1   |
+----------+--------------------------+------------+----------+------------------+
| data-2   | ip-10-1-105.ec2.internal | 10.0.1.105 | data-2   | elastic_data_2   |
+----------+--------------------------+------------+----------+------------------+
| data-3   | ip-10-1-106.ec2.internal | 10.0.1.106 | data-3   | elastic_data_3   |
+----------+--------------------------+------------+----------+------------------+

The built-in user passwords for Elasticsearch will need to be set after enabling transport network encryption using the elasticsearch-setup-passwords tool. For this, set each built-in user's password as follows:

+------------------------+-------------------------------+
| User                   | Password                      |
+------------------------+-------------------------------+
| elastic                | la_elastic_409                |
+------------------------+-------------------------------+
| apm_system             | la_apm_system_409             |
+------------------------+-------------------------------+
| kibana                 | la_kibana_409                 |
+------------------------+-------------------------------+
| logstash_system        | la_logstash_system_409        |
+------------------------+-------------------------------+
| beats_system           | la_beats_system_409           |
+------------------------+-------------------------------+
| remote_monitoring_user | la_remote_monitoring_user_409 |
+------------------------+-------------------------------+

Important: Don’t forget to add the password you used to protect the certificate package to the local Elasticsearch keystore on each node as both the truststore and keystore secure password.

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?