Hello community! Sorry for the potential off-topic
I am currently working on an Angular project in which our users would upload a picture to our back-end via API Gateway and lambda functions. Our back-end (an EC2 instance, for example) will then transform the picture and save it in an S3 bucket ready to be displayed to the user. This operation might take up to 30 seconds.
I was wondering what is the "state of the art" serverless way of having the front-end wait for the back-end to complete the operation and "immediately" show it to the end-user. In essence, how do I "subscribe" and wait for the server to complete the operation via API Gateway and lambda functions?
I have thought of the following options but none of them appears to be used in this context:
1) Have the front-end call regularly (e.g. every second) the API checking for the specific picture id in the S3 bucket via a lambda function (i.e.: GET /obtain-picture/{id} and wait until a 200 is received and ignore the 404 responses.
2) Have the back-end push a message "complete for id {id}" to an SQS queue. Then, the front-end would pull regularly (e.g. every second) the SQS queue (via a lambda function that would only extract the appropriate information) and check if the pulled message contains the relevant ID. Again, I am not very sure on how scalable this would be.
3) Use a "push-like" service such as SNS and wait for the server to push a message, but I am unsure whether this is even feasible based on the very little resources found.
4) I have read about "asynchronous" lambda calls, but I am unsure on how this would work and how the server could put a lambda function "on-hold" meaning I wouldn’t be billed for it. Would the following link be a solution:
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-integration-async.html
Is there a better way to have the website "wait" until the server "notifies" it that it has completed its processing? In this sense, I am mostly thinking about websites where we would upload a profile picture (LinkedIn), and after loading (and certainly doing a lot of processing), the Website shows the formatted profile picture.
Any relevant resources or tips would be very much appreciated, even if not very detailed
N.B: I am aware of the possibility to "subscribe" and wait for a server response via Angular, but I am not sure how "subscribing" to a lambda function (via API Gateway) stop it from running for too long (I’d like to avoid "unoptimized" billing).
Thank you very much in advance and apologies in case this question is a duplicate.
2 Answers
Hi J,
I was leaving this question alone as I personally don’t have any advise, but maybe that other forum members had some experience in this area. Doesn’t seem like it though.
In general, I’m not a fan of the "please stand by" type of feedback loops for web-based apps. Leaves too much possibility for loosing the connection of something strange happening in that process like a delay for warm-up in the Lambda function. For example, with LinkedIn, if you upload a video, they perform some transcribing I guess and notify you via push notification to the mobile app that your video post is ready.
That said, there are all sorts of new things coming out with the proliferation of PWAs. Sorry for the non-answer.
–Scott
There’s a good re:Invent session that describes some async / callback patterns and pro’s and con’s. Definitely have a look to reinforce your ideas and see what alternatives are possible. The patterns more towards the end, but it’s a good video entirely: https://youtu.be/dzU_WjobaRA?t=2399
One of the options also includes doing the async REST request and then opening a WebSocket in which the backend can reply back on when its done. (the example uses StepFunctions to check if both the connection is opened based on the unique client ID and the backend processing is done).