Those who have used CodePipeline, know the benefits of the service and how easy it can be to setup and run a pipeline. For those new to CodePipeline, it is AWS’s take on a continuous delivery service. It provides the tools to model, visualize, and automate the software release process.CodePipeline, by default, comes with six action “categories” that can be added to any stage in the pipeline. These categories include 15 providers, comprised of both AWS and third party services:
- Approval: Manual
- Source: AWS CodeCommit, AWS S3, GitHub
- Build: Solano CI, Jenkins
- Test: BlazeMeter, RunScope, Ghost Inspector, HPE StormRunner Load, Apica Loadtest, Jenkins
- Deploy: AWS OpsWorks, AWS Elastic BeanStalk, AWS CodeDeploy
- Invoke: AWS Lambda
This list is no slouch! Many of the heavy hitters are here, and these default options should be enough for most individuals or organizations.However, what if your software requires a special action provider? Something that had been tailored specifically to build or test your unique application? Perusing CodePipeline in the AWS Console cannot provide you with any insight into how (or even if) you can create, add, and implement your own custom action that uses your custom provider. Only if you dive deep into the AWS CodePipeline documentation, do you learn that you can create your own actions using a custom provider, but it can only be created via the AWS Command Line interface (CLI). And once you get the custom action created, you still need to configure your custom provider (the “job worker”) to communicate with AWS.
Scenario: Add a Custom Actions to Call Upon a Custom Build Application
Let’s say that you have a custom-built application for your software — something like Jenkins — that was built in-house specifically to build the software you want to release.
Step 1: Create a Custom Action
- In a text editor download, and copy and paste in the AWS Custom Action sample template. (Custom Action Template)
- Edit/Configure the template: The image below shows the fields in the template that need to be edited, what the fields are, and how they are reflected once the action has been created.
- Save the template as a .json file.
- Upload the file to AWS using the command:
aws codepipeline create-custom-action-type --cli-input-json file://<FILE_NAME>.json
The custom action will then be available for your to add in the AWS Console (you may have to exit our of your pipeline, refresh the page, then go back into the pipeline for the custom action to appear in the drop down menus).
Step 2: Configure Your Custom Build Application for use with CodePipeline
Moving forward, the custom build application will be known as a “job worker.”The job worker must be configured to continuously poll CodePipeline for a “job request.” When a “job worker” detects a “job request” it must be set up to complete the following:
- Acknowledge (to CodePipeline) that it has detected the job request
- Pull the target (input) artifacts from the pipeline’s S3 bucket
- Execute the custom job (perform actions on the artifacts)
- Push the (output) artifacts back to the pipeline’s S3 bucket
- Return the job results to CodePipeline (succeeded or failed)
There are three main tasks that you need to accomplish:
- Give the job worker permissions to access CodePipeline:
- The easiest way to do this is to create an instance role, and attached the “AWSCodePipelineFullAccess” to it.
- Launch an EC2, and assign the instance role to it (at launch).
- Install your custom build application on the EC2 instance
- Setup and configure the API calls:
- For polling: PollForJobs
- For job acknowledgement: AcknowledgeJob
- For failure: PutJobFailureResult
- For updates & succeeded status: PutJobSuccessResult
- The continuation token allows you to update the action in the AWS console to include the ExternalExecutionID that will populate the link in “executionUrlTemplate”
- Setup API calls so that your “job worker” can communicate with CodePipeline & accomplish the needed tasks (AWS Codepipeline API Documentation)
- Configure an Amazon S3 Client to retrieve and put artifacts:
- To download pipeline artifacts from S3, you must create a S3 Client that uses “signature version 4 signing” (Sig V4).
- To upload pipeline artifacts to S3, you must configure the S3 PutObject request to use encryption.
- Currently only SSE-KMS is supported for encryption. (AWS S3 API Documentation, AWS SSE-KMS Documentation)
Step 3: Add the Custom action to your pipeline & select the job worker as your custom provider
Step 4: Run the pipeline!
For a detailed, video walkthrough of this scenario – head on over to LinuxAcademy.com and checkout the “Manage & Deploy Code with AWS Developer Tools” course.