DynamoDB has a cumbersome ‘type system’
When using DynamoDB, the API can take a little getting used to. One of the more annoying parts of the API is how attribute data types are specified. Each Item Attribute is itself a map with only one key, where the key signifies the datatype, like the following:
This means you can’t just access an attribute by saying
item.attribute, you must say
item.attribute.TYPE. Well, what if you don’t know the type? You could make the argument ‘This attribute is always a string, so let’s just assume it’s a string’, but you never know when someone in the future may change it’s type, either purposefully or in error.
AWS SDKs to the rescue!
Compare the two gists below:
The above is way easier to manage than the ‘raw API’ as you see below.
Now this is all well and good, there is never a time when you wouldn’t use the DocumentClient …unless you’re using DynamoDB Streams
DynamoDB Streams is a feature where you can stream changes off your DynamoDB table. This allows you to use the table itself as a source for events in an asynchronous manner, with other benefits that you get from having a partition-ordered stream of changes from your DynamoDB table.
The problem is, when you use AWS Lambda to poll your streams, you lose the benefits of the DocumentClient! You are no longer calling DynamoDB at all from your code. Your Lambda is invoked with the body from the stream. You never are ‘given the chance’ to insert the DocumentClient in the right place to do this JS object translation for you.
The good news is … we can tap into this translator directly!
I did some digging and learned how to pull the nice object translation layer out of the DocumentClient and invoke it directly. See the gist below:
Streaming Code Running in Lambda
That’s all there is to it! Your DynamoDB Streaming code running in Lambda now has the niceties of JS objects as provided by the DocumentClient.
- This is in no way using public APIs from the SDK. These could change or go away at anytime. I protect from this issue by always including my own AWS SDK in my build — so Lambda SDK upgrades won’t break me — and thus my local integration tests will catch a SDK break.
- You may be able to use a single translateOutput call using the getRecords shape directly. I didn’t try very hard to make this work.
That’s it for this post. I hope this helps you cut down on a lot of the boilerplate in your DynamoDB Streaming listeners.
Get the skills you need for a better career.
Master modern tech skills, get certified, and level up your career. Whether you’re starting out or a seasoned pro, you can learn by doing and advance your career in cloud with ACG.