Deploying AWS CloudFormation templates from CircleCI
By Martijn Storck
CircleCI provides several official orbs for interacting with Amazon Web Services, including ECR (the container registry), ECS (the container runtime) and S3. There is no official CloudFormation orb for CircleCI and the community options seem poorly maintained. Fortunately, you don’t really need an orb since CloudFormation stacks are easily updated using the AWS CLI.
For a recent project I needed (wanted) to deploy updates to ECS not by updating the container image using the aws-ecs orb as I usually do, but by deploying the complete CloudFormation stack from CI. The benefit of this is that you can deploy changes to the stack (such as the ENV vars from your container) by changing the CloudFormation template in Git. This is similar to how deployments to Kubernetes are often handled; storing the deployment manifest in Git and applying that on every CI deploy.
Just show me the code, please!
Ok, here’s what you do. Create a context in your CircleCI configuration that contains the necessary ENV variables for the AWS CLI to connect to your account and region:
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_DEFAULT_REGION
I suggest naming this context after the account and region, like aws-12345-us-east-2. Then,
configure your .circleci/config.yml
as follows:
version: 2.1
orbs:
aws-cli: circleci/aws-cli@1.4.0
jobs:
deploy-cfn:
parameters:
stack-name:
description: Name of the CloudFormation stack
type: string
executor: aws-cli/default
steps:
- checkout
- aws-cli/setup
- run:
name: Deploy Cloudformation Template with new Docker image
command: |
aws cloudformation deploy \
--template cfn/my-template.yml \
--stack << parameters.stack-name >> \
--parameter-overrides ImageTag=${CIRCLE_SHA1}
workflows:
deploy:
- … # build image and push to ECR with ${CIRCLE_SHA1} tag
- deploy-cfn:
context:
- aws-12345-us-east-2
stack-name: My-Application
The key here is the ImageTag parameter in the stack. I found this the best way to allow the
user to specify the tag to deploy for a Docker image for both manual and automated deploys.
No sed
necessary! We simply use the Sub function to substitute the parameter in the ECS Task
Definition. These are the relevant bits from the CloudFormation template:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
ImageTag:
Type: String
Description: tag of the docker image to deploy in this stack
Default: latest
…
Resources:
TaskDefMyApp:
Type: AWS::ECS::TaskDefinition
Properties:
Family: MyApp
ContainerDefinitions:
- Image: !Sub '12345.dkr.ecr.us-east-2.amazonaws.com/my-app:${ImageTag}'
If you have multiple Task Definitions running the same container image (like web and background worker),
you only have to override the parameter once. This is a lot better than doing multiple
container-image-name-updates
with the aws-ecs orb, as we did before.
Bonus: Linting the CloudFormation template in CircleCI
The CloudFormation update is one of the last steps in what is often a long CI pipeline. To prevent syntax errors in the template from causing disappointment after a long and exciting pipeline run, I recommend adding cfn-lint, the AWS CloudFormation Linter to the pipeline. This can run along your regular tests and will notify you if the template is invalid. This is the job specification:
jobs:
cfn-lint:
docker:
- image: cimg/python:3.9
steps:
- checkout
- run:
name: Validate CloudFormation template
command: pip install cfn-lint && cfn-lint cfn/my-template.yml
Make sure you add this job to the requires
section of your deploy-cfn
step.
Conclusion: No orbs needed!
Not reaching for community orbs helps keep your infrastructure safe and makes CircleCI configuration files easy to understand. Deploying CloudFormation templates from CircleCI is easily achieved using a single call to the AWS CLI and can be easily adapted to complex workflows.