This lab presents an application that is struggling under load due to a low read/write provisioned capacity. We navigate through the process of provisioning an ElastiCache cluster so we can implement a simplistic cache in front of the database. Basic knowledge of the Linux CLI, file commands, and Vim editor is expected.
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Measure the Baseline Performance
Start by taking a baseline performance measurement of the application. We will compare this with the Redis performance after our changes.
Log in to the AWS Management Console using the instructions above.
Search for RDS using the search bar. Select Databases on the left-hand side menu.
Select the instance under the DB identifier column.
Under Endpoint copy the endpoint URL. It will be used momentarily.
Switch over to a terminal and login to the bastion host (aka Cloud Server) using the credentials above.
Download the application file (
app.py) and its configuration files (database.iniandapp.sql) from the Git repository;
This can be downloaded to the bastion host with the following command:git clone https://github.com/linuxacademy/content-aws-database-specialty.gitCreate the
appdirectory:mkdir /home/cloud_user/app/Copy these three files into a directory named
app:cp /home/cloud_user/content-aws-database-specialty/S08_Using Elasticache to Improve Database Performance/{app.py,app.sql,database.ini} /home/cloud_user/app/Edit the database configuration:
cd appvim database.iniSet the hostname to the endpoint (copied earlier in this video and also paste in the password to the bastion host:
host=<YourRDSEndpoint> password=<BastionHostPW>Save and quit with
:wq.Additionally, save the hostname in the
PGHOSTvariable:export PGHOST=<YourRDSEndpoint>Ensure you can connect to the database with the command below and enter in the bastion password:<br>
psql -U postgres -h $PGHOSTQuit with
q.Install the SQL files for the application and, once again, input the bastion password:
psql -U postgres -h $PGHOST -f app.sqlSpecify the Python version for the application:
python2.7 app.pySwitch to a browser and input the bastion host IP into the URL bar, while navigating to the
appdirectory, like so:
bastionip/appIf successful, you’ll see the Postgres version displayed on the screen.
Head over to your terminal and open a new window or tab and save the application URL as an environmental variable:
export APP_URL=http://<bastionIP>/appTest the URL using the cURL command:
curl -L $APP_URLNote: Need to install cURL on Mac? Try this.
Create the benchmark by using a
forloop:time (for i in {1..5}; do curl -L $APP_URL;echo n;done)Once completed, note the seconds it took for these requests. In our lesson, it took 25.462 seconds.
- Deploy the Redis Cluster
Deploy a new Redis cluster to use as our cache.
In the AWS Management Console, go back to the home screen, (select AWS logo) and type in the search bar VPC.
Click on Security Groups in the left-hand side menu.
Note the first security group under the security group ID, when clicked will show EC2SecurityGroup(under details).
Select Create Security Group.
Fill out the basic details of the security group:
- Security group name: CacheSG
- Description: cache security group
In the Inbound rule section, click Add rule. Change Port range to 6379.
Select the security group notated in step 3 for the box next to Source.
In the Tags section, add a tag with the key of name and value of Cache SG.
Click Create Security Group. A confirmation of creation will display on the screen.
Go back to the AWS console’s home screen, and search the for ElastiCache service in the search bar.
Click Get Started Now button.
Set:
- Cluster Engine: Redis
- Name:Cache
- Description:cache aside for app
- Node type: t2 –> cache.t2.micro<br>
- Select Save.
- Number of replicas: 1<br>
Name of Advanced Redis settings:
- Name:cache
- Description: cache subnet group
- Subnets: Select two subnets that are different Availability Zones.
Under the Security section:
- Security group: change the default to Cache SG
Under the Backup section:
- Enable backups: Uncheck (Disable)
- Select the Create button.<br>
The cluster will take some time to create (5 ~7mins, so take a stretch!). Once complete, click the dropdown arrow next to the cluster name.
Copy the Primary Endpoint.
Switch back to the bastion host terminal. Exit the Python app (
ctrl + c) and clear the screen (typeclear).Check the connection to the cluster by using the Python REPL:
python2.7>>> import redis >>> client = redis.Redis.from_url('redis://<PrimaryEndpoint>') >>> client.ping()Exit the Python REPL (
ctrl +d).
- Update the Application and Measure its Performance
Lastly, we update the application to use the Redis cluster as a "cache-aside" and re-measure the application’s performance.
In the bastion host terminal, open the
app.pyfile.vim app.pyWe alter the
fetchmethod in this file to utilize the cache. The full version of this snippet is GitHub files we downloaded earlier, namedredis-app.py.... def fetch(sql): ttl = 10 try: params = config(section='redis') cache = redis.Redis.from_url(params['redis_url']) results = cache.get(sql) if result: return result else: # connect to database listed in datbase.ini conn = connect() cur = conn.cursor() cur.execute(sql) # fetch one row result = cur.fetchone() print('Closing connection to database...') cur.close() conn.close() # cache results cache.setex(sql, ttl, ''.join(results)) return result except(Exception, psycopg2.DatabaseError) as error; print(error) ...Save the file with
:wq.Add the Redis URL to the bottom of the
database.iniconfiguration file.vim database.ini[redis] redis_url=redis://<PrimaryEndpoint>Save the file with
:wq.Run the app:
python2.7 app.pyGo back to the browser with the bastion host URL and refresh the screen multiple times, around five.
Head to the terminal on your local machine, and run the
forloop once more:time (for i in {1..5}; do curl -L $APP_URL;echo n;done)Note the new total time. This will be significantly less time than the original total time.
