I have ECS Task set as a target on a CloudWatch Event rule that invokes on the below S3 Event Pattern.
The rule invokes OK on a PUT operation in a S3 bucket, and starts the ECS Task that is set as its target.
The Task reaches RUNNING state… and remains in RUNNING state until it is stopped. I use the CLI to stop the task. Also, this task is not part of a ECS Service, but a stand-alone task intended to do a specific task.
Is there a way to self-destruct the Task after it reaches the RUNNING state and does the intended work? I could wait for 30mins or even a few hours… but ultimately the tasks needs to STOP by itself.
This becomes particularly difficult to manage when there are 1000s of S3 PUT operations that invoke the CloudWatch rule that in-turn starts 1000s of tasks. I am looking for somehow stopping these tasks after they reach the RUNNING state and finish the intended work.
Any suggestions?
1 Answers
Hi Phani,
You might have already thought about this option, but would Lambda be an option or maybe multiple Lambda via Step Functions? Seems like you have a more event-driven need. I’ve always viewed ECS tasks as more long-running microservice type things that should be up perpetually. Lambda is more appropriate for short-duration actions.
Alternatively, maybe you could trigger a lambda function to terminate the ECS task as part of the last step in the process?
Just thoughts…
–Scott
Hi Scott — thanks. Yes Lambda is more appropriate for such an action… however, the requirement is to retrieve stream of data from s3 and store in an on-prem data-store. And my understanding is that Lambda cannot handle large data-sets.
And yeah, this was how I thought it through as well — trigger a lambda function to terminate the ECS task as part of the last step in the process.