Professional Level

Managing Indices with Index Lifecycle Management (ILM) in Elasticsearch 7.13

1 hour

3 Learning Objectives

Efficiently ingesting continuously generated data 24/7 is one of the many things Elasticsearch is very good at. But eventually, you’re going to run out of space. This is why it is important to leverage the index lifecycle management (ILM) feature in Elasticsearch. In this hands-on lab, you will create ILM policies in Elasticsearch to optimize and move cold data to slower nodes and to retire old data to free up space.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Create the audit_policy ILM Policy

Create the new audit_policy index lifecycle management (ILM) policy.
Configure the cold phase with a minimum age of 7 days and the freeze and readonly actions.
Configure the delete phase with a minimum age of 365 days and the delete action.

Create the audit_template Index Template

Create the new audit_template index template.
Configure the index pattern to match any index that starts with audit-.
Configure the index template to create indices with 1 primary and 0 replica shards.
Configure the index template to create indices that use the audit_policy ILM policy.

Create and Verify the First audit-DD-MM-YYYY Index

Create the first audit-DD-MM-YYYY index using the current date.
Verify that the index was created under the audit_template template and has the audit_policy ILM policy.

Additional Resources

Logging In to the Elastic Environment

Open a new browser tab and navigate to the public IP address of the es1 node provided on the lab page (e.g., http://public_ip).
Log in using the username elastic and password elastic_acg.

Lab Scenario

You are a cyber security analyst who is in charge of collecting and maintaining an audit log of user access changes at your company. For this, you will use Elasticsearch to store the audit data in a daily index with the format audit-MM-DD-YYY. However, you need to first configure an index lifecycle management (ILM) policy for the audit data as follows:

Data is first ingested as hot data.
After 7 Days, the data is converted to cold data with a frozen and readonly index.
After 365 days, the data is deleted.

Once the ILM policy is created, you need to create the audit_template index template to match any index with the naming pattern of audit-, and apply the audit_policy ILM policy. Also, because you are working on a single-node cluster, the audit_template index template should be configured to create indices with 1 primary and 0 replica shards in order to maintain a green cluster state. Lastly, create and verify the first audit index with the current date using the audit-MM-DD-YYY format.

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started

Who’s going to be learning?

Myself My Organization