Professional Level

Configure Shard Allocation and Forced Awareness in Elasticsearch

Pricing
1.5 hours
  • 5 Learning Objectives

About this Hands-on Lab

With Elasticsearch, you have the potential to scale your cluster to be highly available. To ensure your data survives a full zone or rack failure, you can spread your data across multiple zones and use shard allocation awareness to ensure that all replica shards are allocated to a different zone than that of their primary shard. This allows another level of fault tolerance beyond just node failure. In this hands-on lab, you are given the opportunity to exercise the following:

* Apply node attributes to each data node in a cluster.
* Configure shard allocation awareness for a cluster in multiple zones.
* Test shard allocation awareness behavior by stopping all nodes in a zone.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Configure nodes data-1 and data-2 to have a “zone” node attribute with a value of “1”.

Using the Secure Shell (SSH), log in to the data-1 and data-2 nodes as cloud_user via the public IP address.

Become the elastic user with:

sudo su - elastic

In the /home/elastic/elasticsearch/config/elasticsearch.yml file, change the following line:

#node.attr.rack: r1

to

node.attr.zone: 1

Restart the elasticsearch node with:

pkill -F /home/elastic/elasticsearch/pid
/home/elastic/elasticsearch/bin/elasticsearch -d -p pid
Configure nodes data-3 and data-4 to have a “zone” node attribute with a value of “2”.

Using the Secure Shell (SSH), log in to the data-3 and data-4 nodes as cloud_user via the public IP address.

Become the elastic user with:

sudo su - elastic

In the /home/elastic/elasticsearch/config/elasticsearch.yml file, change the following line:

#node.attr.rack: r1

to

node.attr.zone: 2

Restart the elasticsearch node with:

pkill -F /home/elastic/elasticsearch/pid
/home/elastic/elasticsearch/bin/elasticsearch -d -p pid
Enable shard allocation awareness for the cluster per instructions.

Use the Kibana console tool to execute the following:

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.awareness.attributes": "zone",
    "cluster.routing.allocation.awareness.force.zone.values": "1,2"
  }
}
Stop Elasticsearch on nodes data-3 and data-4.

Using the Secure Shell (SSH), log in to the data-3 and data-4 nodes as cloud_user via the public IP address.

Become the elastic user with:

sudo su - elastic

Stop the elasticsearch node with:

pkill -F /home/elastic/elasticsearch/pid
Ensure that all primary shards are assigned and all replica shards remain unassigned.

Use the Kibana console tool to execute the following:

GET _cat/shards?v
Contact SalesSee Pricing

Additional Resources

You work as an Elasticsearch administrator for a bank. You manage an Elasticsearch cluster that is spread across two zones and wish to enable shard allocation awareness to ensure that replica shards prefer allocation to different zones than their primary shard. This way, if one of your zones where to experience downtime, a complete copy of your data would always exist in the other zone.

In the event of a zone failure, you want to make sure that your remaining zone is not overwhelmed with replica shard re-allocation. Therefore, you want to configure shard allocation awareness to not re-allocate missing replica shards if there is only one zone available.

Nodes data-1 and data-2 should have a zone node attribute with a value of 1. Nodes data-3 and data-4 should have a zone node attribute with a value of 2. Once each data node is configured with the corresponding zone, you will need to restart Elasticsearch on these nodes and then configure the cluster for shard allocation awareness. Lastly, you will need to test your configuration by stopping all nodes belonging to zone 2 and verifying that all primary shards are assigned and all replica shards remain unassigned as to not overwhelm the last available zone.

To use Kibana, navigate to the public IP address of the coordinator-1 node in your web browser and login with:

  • Username: elastic
  • Password: la_elastic_409

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Don't have an account?

Get Started
Who’s going to be learning?
MyselfMy Organization