Deploying AWS CloudFormation templates from CircleCI

CircleCI provides several official orbs for interacting with Amazon Web Services, including ECR (the container registry), ECS (the container runtime) and S3. There is no official CloudFormation orb for CircleCI and the community options seem poorly maintained. Fortunately, you don’t really need an orb since CloudFormation stacks are easily updated using the AWS CLI.

For a recent project I needed (wanted) to deploy updates to ECS not by updating the container image using the aws-ecs orb as I usually do, but by deploying the complete CloudFormation stack from CI. The benefit of this is that you can deploy changes to the stack (such as the ENV vars from your container) by changing the CloudFormation template in Git. This is similar to how deployments to Kubernetes are often handled; storing the deployment manifest in Git and applying that on every CI deploy.

Just show me the code, please!

Ok, here’s what you do. Create a context in your CircleCI configuration that contains the necessary ENV variables for the AWS CLI to connect to your account and region:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_DEFAULT_REGION

I suggest naming this context after the account and region, like aws-12345-us-east-2. Then, configure your .circleci/config.yml as follows:

version: 2.1

orbs:
  aws-cli: circleci/aws-cli@1.4.0

jobs:
  deploy-cfn:
    parameters:
      stack-name:
        description: Name of the CloudFormation stack
        type: string
    executor: aws-cli/default
    steps:
      - checkout
      - aws-cli/setup
      - run:
          name: Deploy Cloudformation Template with new Docker image
          command: |
            aws cloudformation deploy \
            --template cfn/my-template.yml \
            --stack << parameters.stack-name >> \
            --parameter-overrides ImageTag=${CIRCLE_SHA1}

workflows:
  deploy:
    - … # build image and push to ECR with ${CIRCLE_SHA1} tag
    - deploy-cfn:
      context:
        - aws-12345-us-east-2
      stack-name: My-Application

The key here is the ImageTag parameter in the stack. I found this the best way to allow the user to specify the tag to deploy for a Docker image for both manual and automated deploys. No sed necessary! We simply use the Sub function to substitute the parameter in the ECS Task Definition. These are the relevant bits from the CloudFormation template:

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  ImageTag:
    Type: String
    Description: tag of the docker image to deploy in this stack
    Default: latest
…
Resources:
  TaskDefMyApp:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: MyApp
      ContainerDefinitions:
        - Image: !Sub '12345.dkr.ecr.us-east-2.amazonaws.com/my-app:${ImageTag}'

If you have multiple Task Definitions running the same container image (like web and background worker), you only have to override the parameter once. This is a lot better than doing multiple container-image-name-updates with the aws-ecs orb, as we did before.

Bonus: Linting the CloudFormation template in CircleCI

The CloudFormation update is one of the last steps in what is often a long CI pipeline. To prevent syntax errors in the template from causing disappointment after a long and exciting pipeline run, I recommend adding cfn-lint, the AWS CloudFormation Linter to the pipeline. This can run along your regular tests and will notify you if the template is invalid. This is the job specification:

jobs:
  cfn-lint:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Validate CloudFormation template
          command: pip install cfn-lint && cfn-lint cfn/my-template.yml

Make sure you add this job to the requires section of your deploy-cfn step.

Conclusion: No orbs needed!

Not reaching for community orbs helps keep your infrastructure safe and makes CircleCI configuration files easy to understand. Deploying CloudFormation templates from CircleCI is easily achieved using a single call to the AWS CLI and can be easily adapted to complex workflows.

• AWS
• DevOps