Event-driven architecture with Amazon SQS and AWS Lambda

The centerpiece of event-driven architecture is a queue — made easier now that Amazon SQS can trigger AWS Lambda functions.

You probably know that I am a huge fan of event-driven patterns. Instead of waiting and polling for changes, actions happen when changes in the system occur.

We have talked about this architectural pattern for years — but until recently, it was more of a theoretical idea rather than a simple-to-implement pattern.

However, AWS Lambda brought event-driven compute to everyone and took the event-driven paradigm to a whole new level. Companies like Netflix (Bless) and Capital One (Cloud Custodian) are now using event-driven architectures to perform real-time security, compliance and policies management.

The centerpiece of event-driven architectures is often a queue. On AWS, that central building block is taken care of by Amazon Simple Queue Service (SQS). In order to read information from an SQS queue, your lambda function had to poll for it — until now!

While drinking my coffee, I noticed something interesting in the list of triggers for my test lambda function. I was like — what?? Do I dream?

WHAAAAAT - This showed up in my AWS Console --- Yes my friends! This is the day when SQS becomes a trigger for AWS Lambda. #aws #Serverless pic.twitter.com/b79Ml7yLNv

— Adrian Hornsby @ArcticCircle (@adhorn) June 28, 2018

• First, create an SQS queue. I called my test queue TestMySQSLambda which is the most original name I could come up before two cups of morning coffee.
• I also used the default Standard Queue — which does not guarantee the order of the messages.
• Click Quick-Create Queue — yes, we want it quick!

• Then create an IAM Role for your lambda function to be able to get triggered by SQS. This is a test, so let’s go wild and add that bellow policy to your basic lambda execution role — and save it as SQSLambda role.

• Then log into the AWS Lambda console and let’s create a function from scratch. Again, a very original name is required — MySQSTriggerTest.

• Click createandthen edit the Lambda function — I used the following super advanced python code.

• Now select the Execution role just created before.

• Then select the created SQS trigger from the long list of triggers — it is the last in the list.

• Finally, select the TestMySQSLambda SQS queue previously created.
• Click Add and then Save.
• If everything was done correctly, you should see something like that in the AWS Console — with Saved in green.

• Select Configure test event from the menu as shown below.

• Then search for SQS and you should see the test template as below.

• Click Create and then click Test.

All green — we are good! Now let’s test the whole pipeline by putting data directly into the SQS queue.

• Go into the SQS console and select the queue created before and click Queue Actionsandthen selectSend a Messagefrom the dropdown.

• Type your message of hope to the world here. This is your chance to be a good person today!

• Click Send Message.

Now go check the Cloudwatch logs console and search for aws/lambda/MySQSTriggerTest — you should be able to see the SQS to Lambda trigger!

Niiiice! This was a long-awaited feature which now allows developers to build truly event-driven architecture without needing to poll the SQS queue for data! Can’t wait to see what you are going to build with it :-)

Master modern tech skills, get certified, and level up your career. Whether you’re starting out or a seasoned pro, you can learn by doing and advance your career in cloud with ACG.