Remediating EC2 Auto Scaling Group Modifications with EventBridge

30 minutes
  • 3 Learning Objectives

About this Hands-on Lab

For this lab, we will incorporate a few different services to build a simple monitoring and remediation system for accidental changes made to **production** autoscaling groups, while ignoring **development** changes. The architecture will utilize CloudTrail, EventBridge, EC2 (Autoscaling Groups), and Lambda.

We will find there are two EC2 autoscaling groups that have different `environment` tags attached, **prd** and **dev**. Using these tags in conjunction with EventBridge, we will create an Event Pattern that matches any EC2 termination changes made to any autoscaling group, which then will invoke a custom Lambda function. This Lambda function will complete some operational checks to see if the tags are set to **prd**, and if so, start resetting the desired capacity. If the tags are set to **dev**, then we will only print the instances and environment to the CloudWatch logs.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Create an EventBridge Rule

We want to be able to understand the process of creating an EventBridge rule that watches for event patterns that we specify using available AWS CloudTrail API calls.

We want to look for an Auto Scaling service event called EC2 Instance Termination Successful within the account.

If you need to reference the event pattern for some help, you can find the solution pattern over in our GitHub Repo

Configure a Lambda Target

Use the deployed Lambda function as a target for our EventBridge rule. The Lambda will parse the auto scaling event data that is sent to it and, if the group is tagged as prd, it will correct the changes immediately.

Successfully Reset Production Changes

Once all the pieces are put together, we should now test that the system works.

When you scale down the Development auto scaling group, it should trigger the Lambda and only print logs to CloudWatch.

However, if you scale down the Production auto scaling group, it should trigger the Lambda, which will then do two things:

  1. Reset the baseline capacity to 1 instance
  2. Print the information to CloudWatch logs

Additional Resources

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?