Before we can get hands-on with indexing, searching, and aggregating our data with Elasticsearch, we first need to know how to prepare a system and how to deploy and configure Elasticsearch. In this hands-on lab, you will deploy a 6-node Elasticsearch cluster with a specific set of configuration requirements. Specifically, you will:
* Create an `elastic` user
* Configure the open limit
* Set memory map limits via sysctl
* Deploy Elasticsearch from an archive
* Specify Elasticsearch cluster and node names
* Create custom attributes for Elasticsearch nodes
* Assign Elasticsearch node roles
* Configure the Elasticsearch Java virtual machine (JVM) heap
* Bind Elasticsearch to specific network addresses
* Configure Elasticsearch node discovery
* Configure Elasticsearch cluster bootstrap
* Start Elasticsearch as a daemon
* `curl` the Elasticsearch nodes to check status and configuration
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Prepare each node, create the elastic user, and deploy Elasticsearch.
Create the
elastic
user:sudo useradd elastic
Open the
limits.conf
file as root:sudo vim /etc/security/limits.conf
Add the following line near the bottom:
elastic - nofile 65536
Open the
sysctl.conf
file asroot
:sudo vim /etc/sysctl.conf
Add the following line at the bottom:
vm.max_map_count=262144
Load the new sysctl values:
sudo sysctl -p
Become the
elastic
user:sudo su - elastic
Download the binaries for Elasticsearch 7.2.1 in the
elastic
user’s home directory:curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.2.1-linux-x86_64.tar.gz
Unpack the archive:
tar -xzvf elasticsearch-7.2.1-linux-x86_64.tar.gz
Remove the archive:
rm elasticsearch-7.2.1-linux-x86_64.tar.gz
Rename the unpacked directory:
mv elasticsearch-7.2.1 elasticsearch
- Configure each node’s elasticsearch.yml per instructions.
Log in to each node and become the
elastic
user:sudo su - elastic
Open the
elasticsearch.yml
file:vim /home/elastic/elasticsearch/config/elasticsearch.yml
Change the following line:
#cluster.name: my-application
to
cluster.name: linux_academy
Change the following line on master-1:
#node.name: node-1
to
node.name: master-1
Change the following line on master-2:
#node.name: node-1
to
node.name: master-2
Change the following line on master-3:
#node.name: node-1
to
node.name: master-3
Change the following line on data-1:
#node.name: node-1
to
node.name: data-1
Change the following line on data-2:
#node.name: node-1
to
node.name: data-2
Change the following line on data-3:
#node.name: node-1
to
node.name: data-3
Change the following line on master-1:
#node.attr.rack: r1
to
node.attr.zone: 1
Change the following line on master-2:
#node.attr.rack: r1
to
node.attr.zone: 2
Change the following line on master-3:
#node.attr.rack: r1
to
node.attr.zone: 3
Change the following line on data-1:
#node.attr.rack: r1
to
node.attr.zone: 1
Add the following line on data-1:
node.attr.temp: hot
Change the following line on data-2:
#node.attr.rack: r1
to
node.attr.zone: 2
Add the following line on data-2:
node.attr.temp: hot
Change the following line on data-3:
#node.attr.rack: r1
to
node.attr.zone: 3
Add the following line on data-3:
node.attr.temp: warm
Add the following lines on master-1:
node.master: true node.data: false node.ingest: false
Add the following lines on master-2:
node.master: true node.data: false node.ingest: false
Add the following lines on master-3:
node.master: true node.data: false node.ingest: false
Add the following lines on data-1:
node.master: false node.data: true node.ingest: true
Add the following lines on data-2:
node.master: false node.data: true node.ingest: true
Add the following lines on data-3:
node.master: false node.data: true node.ingest: false
Change the following on each node:
#network.host: 192.168.0.1
to
network.host: [_local_, _site_]
Change the following on each node:
#discovery.seed_hosts: ["host1", "host2"]
to
discovery.seed_hosts: ["10.0.1.101", "10.0.1.102", "10.0.1.103"]
Change the following on each node:
#cluster.initial_master_nodes: ["node-1", "node-2"]
to
cluster.initial_master_nodes: ["master-1", "master-2", "master-3"]
- Configure the heap for each node per instructions.
Log in to each master node and become the
elastic
user:sudo su - elastic
Open the
jvm.options
file:vim /home/elastic/elasticsearch/config/jvm.options
Change the following lines:
-Xms1g -Xmx1g
to
-Xms768m -Xmx768m
Log in to each data node and become the
elastic
user:sudo su - elastic
Open the
jvm.options
file:vim /home/elastic/elasticsearch/config/jvm.options
Change the following lines:
-Xms1g -Xmx1g
to
-Xms2g -Xmx2g
- Start Elasticsearch as a daemon on each node.
Log in to each node and become the
elastic
user:sudo su - elastic
Switch to the
elasticsearch
directory:cd /home/elastic/elasticsearch
Start Elasticsearch as a daemon:
./bin/elasticsearch -d -p pid
Check the startup process:
less /home/elastic/elasticsearch/logs/linux_academy.log
Check the node configuration:
curl localhost:9200/_cat/nodes?v