Allocate Shards of Indices to Specific Elasticsearch Nodes

1.5 hours
  • 5 Learning Objectives

About this Hands-on Lab

A great way to save money when sizing compute and storage for an Elasticsearch cluster is by using hot-warm architectures. Especially for time series data, hot-warm architectures allow you to allocate your most relevant data (hot data) on your fastest nodes (hot nodes) and your less relevant data (warm data) on your slower nodes (warm nodes). This allows you to scale data retention with cheaper compute and storage and only pay a premium for your hot nodes. All this is made possible in Elasticsearch through the use of node attributes and allocation filtering, which can be used in any case where you want your index allocation to meet specific requirements. In this hands-on lab, you are given the opportunity to exercise the following:

* Apply node attributes to each data node in a cluster
* Configure indexes to allocate to specific nodes based on their attributes

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Configure nodes data-1 and data-2 to have a “temp” node attribute of “hot”.

Using the Secure Shell (SSH), log in to the data-1 and data-2 nodes as cloud_user via the public IP address.

Become the elastic user with:

sudo su - elastic

In the /home/elastic/elasticsearch/config/elasticsearch.yml file, change the following line:

#node.attr.rack: r1

to

node.attr.temp: hot

Restart the elasticsearch node with:

pkill -F /home/elastic/elasticsearch/pid
/home/elastic/elasticsearch/bin/elasticsearch -d -p pid
Configure nodes data-3 and data-4 to have a “temp” node attribute of “warm”.

Using the Secure Shell (SSH), log in to the data-3 and data-4 nodes as cloud_user via the public IP address.

Become the elastic user with:

sudo su - elastic

In the /home/elastic/elasticsearch/config/elasticsearch.yml file, change the following line:

#node.attr.rack: r1

to

node.attr.temp: warm

Restart the elasticsearch node with:

pkill -F /home/elastic/elasticsearch/pid
/home/elastic/elasticsearch/bin/elasticsearch -d -p pid
Allocate the logs-03 index to the hot nodes.

Use the Kibana console tool to execute the following:

PUT logs-03/_settings
{
  "index.routing.allocation.require.temp": "hot"
}
Allocate the logs-02 index to the warm nodes.

Use the Kibana console tool to execute the following:

PUT logs-02/_settings
{
  "index.routing.allocation.require.temp": "warm"
}
Allocate the logs-01 index to the warm nodes.

Use the Kibana console tool to execute the following:

PUT logs-01/_settings
{
  "index.routing.allocation.require.temp": "warm"
}

Additional Resources

You work as an Elasticsearch administrator of a cluster used for log aggregation. In an effort to increase your data retention, you recently added two more data nodes to your cluster and want to establish a hot-warm architecture to facilitate the purchasing of more cheap nodes to be used as warm nodes. You are storing your log data in weekly indexes with the naming scheme of logs-<week_number> and only want the current week's index to be allocated to the hot nodes. The rest of the indexes should be allocated to your warm nodes.

First, you will need to add the temp attribute to your data nodes with a value of hot on data-1 and data-2 and a value of warm on data-3 and data-4. Next, you will need to update the index settings for each index to allocate to the desired nodes. Only the most recent week's index should be allocated to the hot nodes.

To use Kibana, navigate to the public IP address of the coordinator-1 node in your web browser and login with:

  • Username: elastic
  • Password: la_elastic_409

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Get Started
Who’s going to be learning?

How many seats do you need?

  • $499 USD per seat per year
  • Billed Annually
  • Renews in 12 months

Ready to accelerate learning?

For over 25 licenses, a member of our sales team will walk you through a custom tailored solution for your business.


$2,495.00

Checkout
Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!