The centerpiece of event-driven architecture is a queue — made easier now that Amazon SQS can trigger AWS Lambda functions
For those that follow some of my prior posts, you probably know that I am a huge fan of event-driven patterns. Instead of waiting and polling for changes, actions happen when changes in the system occur.
We have talked about this architectural pattern for years — but until recently, it was more of a theoretical idea rather than a simple-to-implement pattern.
However, AWS Lambda brought event-driven compute to everyone and took the event-driven paradigm to a whole new level. Companies like Netflix (Bless) and Capital One (Cloud Custodian) are now using event-driven architectures to perform real-time security, compliance and policies management.
The centerpiece of event-driven architectures is often a queue. On AWS, that central building block is taken care of by Amazon Simple Queue Service (SQS). In order to read information from an SQS queue, your lambda function had to poll for it — until now!
While drinking my coffee, I noticed something interesting in the list of triggers for my test lambda function. I was like — what?? Do I dream?
Let’s try this out!
- First, create an SQS queue. I called my test queue
TestMySQSLambdawhich is the most original name I could come up before two cups of morning coffee.
- I also used the default
Standard Queue— which does not guarantee the order of the messages.
Quick-Create Queue— yes, we want it quick!
- Then create an IAM Role for your lambda function to be able to get triggered by SQS. This is a test, so let’s go wild and add that bellow policy to your basic lambda execution role — and save it as
- Then log into the AWS Lambda console and let’s create a function from scratch. Again, a very original name is required —
createandthen edit the Lambda function — I used the following super advanced python code.
- Now select the
Execution rolejust created before.
- Then select the created SQS trigger from the long list of triggers — it is the last in the list.
- Finally, select the
TestMySQSLambdaSQS queue previously created.
- If everything was done correctly, you should see something like that in the AWS Console — with
Time to test!
Configure test eventfrom the menu as shown below.
- Then search for
SQSand you should see the test template as below.
Createand then click
All green — we are good! Now let’s test the whole pipeline by putting data directly into the SQS queue.
- Go into the SQS console and select the queue created before and click
Queue Actionsandthen select
Send a Messagefrom the dropdown.
- Type your message of hope to the world here. This is your chance to be a good person today!
Now go check the Cloudwatch logs console and search for
aws/lambda/MySQSTriggerTest — you should be able to see the SQS to Lambda trigger!
Niiiice! This was a long-awaited feature which now allows developers to build truly event-driven architecture without needing to poll the SQS queue for data! Can’t wait to see what you are going to build with it 🙂
Have a great day!