Deploy and Configure a Multi-Node Elasticsearch Cluster

3 hours
  • 4 Learning Objectives

About this Hands-on Lab

Before we can get hands-on with indexing, searching, and aggregating our data with Elasticsearch, we first need to know how to prepare a system and how to deploy and configure Elasticsearch. In this hands-on lab, you will deploy a 6-node Elasticsearch cluster with a specific set of configuration requirements. Specifically, you will:

* Create an `elastic` user
* Configure the open limit
* Set memory map limits via sysctl
* Deploy Elasticsearch from an archive
* Specify Elasticsearch cluster and node names
* Create custom attributes for Elasticsearch nodes
* Assign Elasticsearch node roles
* Configure the Elasticsearch Java virtual machine (JVM) heap
* Bind Elasticsearch to specific network addresses
* Configure Elasticsearch node discovery
* Configure Elasticsearch cluster bootstrap
* Start Elasticsearch as a daemon
* `curl` the Elasticsearch nodes to check status and configuration

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Prepare each node, create the elastic user, and deploy Elasticsearch.

Create the elastic user:

sudo useradd elastic

Open the limits.conf file as root:

sudo vim /etc/security/limits.conf

Add the following line near the bottom:

elastic - nofile 65536

Open the sysctl.conf file as root:

sudo vim /etc/sysctl.conf

Add the following line at the bottom:

vm.max_map_count=262144

Load the new sysctl values:

sudo sysctl -p

Become the elastic user:

sudo su - elastic

Download the binaries for Elasticsearch 7.2.1 in the elastic user’s home directory:

curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.2.1-linux-x86_64.tar.gz

Unpack the archive:

tar -xzvf elasticsearch-7.2.1-linux-x86_64.tar.gz

Remove the archive:

rm elasticsearch-7.2.1-linux-x86_64.tar.gz

Rename the unpacked directory:

mv elasticsearch-7.2.1 elasticsearch
Configure each node’s elasticsearch.yml per instructions.

Log in to each node and become the elastic user:

sudo su - elastic

Open the elasticsearch.yml file:

vim /home/elastic/elasticsearch/config/elasticsearch.yml

Change the following line:

#cluster.name: my-application

to

cluster.name: linux_academy

Change the following line on master-1:

#node.name: node-1

to

node.name: master-1

Change the following line on master-2:

#node.name: node-1

to

node.name: master-2

Change the following line on master-3:

#node.name: node-1

to

node.name: master-3

Change the following line on data-1:

#node.name: node-1

to

node.name: data-1

Change the following line on data-2:

#node.name: node-1

to

node.name: data-2

Change the following line on data-3:

#node.name: node-1

to

node.name: data-3

Change the following line on master-1:

#node.attr.rack: r1

to

node.attr.zone: 1

Change the following line on master-2:

#node.attr.rack: r1

to

node.attr.zone: 2

Change the following line on master-3:

#node.attr.rack: r1

to

node.attr.zone: 3

Change the following line on data-1:

#node.attr.rack: r1

to

node.attr.zone: 1

Add the following line on data-1:

node.attr.temp: hot

Change the following line on data-2:

#node.attr.rack: r1

to

node.attr.zone: 2

Add the following line on data-2:

node.attr.temp: hot

Change the following line on data-3:

#node.attr.rack: r1

to

node.attr.zone: 3

Add the following line on data-3:

node.attr.temp: warm

Add the following lines on master-1:

node.master: true
node.data: false
node.ingest: false

Add the following lines on master-2:

node.master: true
node.data: false
node.ingest: false

Add the following lines on master-3:

node.master: true
node.data: false
node.ingest: false

Add the following lines on data-1:

node.master: false
node.data: true
node.ingest: true

Add the following lines on data-2:

node.master: false
node.data: true
node.ingest: true

Add the following lines on data-3:

node.master: false
node.data: true
node.ingest: false

Change the following on each node:

#network.host: 192.168.0.1

to

network.host: [_local_, _site_]

Change the following on each node:

#discovery.seed_hosts: ["host1", "host2"]

to

discovery.seed_hosts: ["10.0.1.101", "10.0.1.102", "10.0.1.103"]

Change the following on each node:

#cluster.initial_master_nodes: ["node-1", "node-2"]

to

cluster.initial_master_nodes: ["master-1", "master-2", "master-3"]
Configure the heap for each node per instructions.

Log in to each master node and become the elastic user:

sudo su - elastic

Open the jvm.options file:

vim /home/elastic/elasticsearch/config/jvm.options

Change the following lines:

-Xms1g
-Xmx1g

to

-Xms768m
-Xmx768m

Log in to each data node and become the elastic user:

sudo su - elastic

Open the jvm.options file:

vim /home/elastic/elasticsearch/config/jvm.options

Change the following lines:

-Xms1g
-Xmx1g

to

-Xms2g
-Xmx2g
Start Elasticsearch as a daemon on each node.

Log in to each node and become the elastic user:

sudo su - elastic

Switch to the elasticsearch directory:

cd /home/elastic/elasticsearch

Start Elasticsearch as a daemon:

./bin/elasticsearch -d -p pid

Check the startup process:

less /home/elastic/elasticsearch/logs/linux_academy.log

Check the node configuration:

curl localhost:9200/_cat/nodes?v

Additional Resources

You are a system administrator who has been asked to deploy a 6-node Elasticsearch cluster with very specific configuration requirements:

  • You will need to create an elastic user and then install Elasticsearch version 7.2.1 from an archive at /home/elastic/elasticsearch.
  • Each Elasticsearch instance will need to listen on both the local and site-local addresses.
  • Configure each node as outlined in the table below.
+----------+-----------+-------------------+--------------+----------+
| Server   | Node Name | Attributes        | Roles        | JVM Heap |
+----------+-----------+-------------------+--------------+----------+
| master-1 | master-1  | zone=1            | master       | 768m     |
+----------+-----------+-------------------+--------------+----------+
| master-2 | master-2  | zone=2            | master       | 768m     |
+----------+-----------+-------------------+--------------+----------+
| master-3 | master-3  | zone=3            | master       | 768m     |
+----------+-----------+-------------------+--------------+----------+
| data-1   | data-1    | zone=1, temp=hot  | data, ingest | 2g       |
+----------+-----------+-------------------+--------------+----------+
| data-2   | data-2    | zone=2, temp=hot  | data, ingest | 2g       |
+----------+-----------+-------------------+--------------+----------+
| data-3   | data-3    | zone=3, temp=warm | data         | 2g       |
+----------+-----------+-------------------+--------------+----------+

Don’t forget the prerequisites for Elasticsearch. Specifically, you will need to persistently increase the max open file limit (nofile) for the elastic user to 65536. Lastly, you will need to persistently increase the max map count for processes (vm.max_map_count) to 262144 via sysctl.

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?