Share on facebook
Share on twitter
Share on linkedin

Event-driven architecture with Amazon SQS and AWS Lambda

A Cloud Guru News
A Cloud Guru News

The centerpiece of event-driven architecture is a queue — made easier now that Amazon SQS can trigger AWS Lambda functions

For those that follow some of my prior posts, you probably know that I am a huge fan of event-driven patterns. Instead of waiting and polling for changes, actions happen when changes in the system occur.

We have talked about this architectural pattern for years — but until recently, it was more of a theoretical idea rather than a simple-to-implement pattern.

However, AWS Lambda brought event-driven compute to everyone and took the event-driven paradigm to a whole new level. Companies like Netflix (Bless) and Capital One (Cloud Custodian) are now using event-driven architectures to perform real-time security, compliance and policies management.

Event-driven patterns

The centerpiece of event-driven architectures is often a queue. On AWS, that central building block is taken care of by Amazon Simple Queue Service (SQS). In order to read information from an SQS queue, your lambda function had to poll for it — until now!

While drinking my coffee, I noticed something interesting in the list of triggers for my test lambda function. I was like — what?? Do I dream?

Let’s try this out!

  • First, create an SQS queue. I called my test queue TestMySQSLambda which is the most original name I could come up before two cups of morning coffee.
  • I also used the default Standard Queue — which does not guarantee the order of the messages.
  • Click Quick-Create Queue — yes, we want it quick!
  • Then create an IAM Role for your lambda function to be able to get triggered by SQS. This is a test, so let’s go wild and add that bellow policy to your basic lambda execution role — and save it as SQSLambda role.
  • Then log into the AWS Lambda console and let’s create a function from scratch. Again, a very original name is required — MySQSTriggerTest.
  • Click createandthen edit the Lambda function — I used the following super advanced python code.
  • Now select the Execution role just created before.
  • Then select the created SQS trigger from the long list of triggers — it is the last in the list.
  • Finally, select the TestMySQSLambda SQS queue previously created.
  • Click Add and then Save.
  • If everything was done correctly, you should see something like that in the AWS Console — with Saved in green.

Time to test!

  • Select Configure test event from the menu as shown below.
  • Then search for SQS and you should see the test template as below.
  • Click Create and then click Test.

All green — we are good! Now let’s test the whole pipeline by putting data directly into the SQS queue.

  • Go into the SQS console and select the queue created before and click Queue Actionsandthen selectSend a Messagefrom the dropdown.
  • Type your message of hope to the world here. This is your chance to be a good person today!
  • Click Send Message.

Now go check the Cloudwatch logs console and search for aws/lambda/MySQSTriggerTest — you should be able to see the SQS to Lambda trigger!

Niiiice! This was a long-awaited feature which now allows developers to build truly event-driven architecture without needing to poll the SQS queue for data! Can’t wait to see what you are going to build with it 🙂

Have a great day!

-Adrian

Recommended

Get more insights, news, and assorted awesomeness around all things cloud learning.

Get Started
Who’s going to be learning?
Sign In
Welcome Back!
Thanks for reaching out!

You’ll hear from us shortly. In the meantime, why not check out what our customers have to say about ACG?

How many seats do you need?

  • $499 USD per seat per year
  • Billed Annually
  • Renews in 12 months

Ready to accelerate learning?

For over 25 licenses, a member of our sales team will walk you through a custom tailored solution for your business.


$2,495.00

Checkout