Continuous Integration and Deployment for Rails using CircleCI

Continuous integration and delivery pipeline can have a significant impact the dev team’s productivity and stability of production releases. In this tutorial, I describe how to automate testing, security checks, and deployments for Ruby on Rails apps using CircleCI. I cover a basic CI setup as well as more advanced features like concurrent specs, dependencies caching, NodeJS/Webpack setup, Heroku deployments, and GitHub integration.

It’s a quickstart introduction and high-level overview of CircleCI continuous integration setup. For more in-depth info you can check out the official docs.

The post is written in the context of Ruby on Rails tech stack, but with small changes, the same approach can be used with any language.

Advert

My RDS migration guide is now out!

This is the eBook that I wish existed when I was first tasked with moving the Heroku database to AWS as a developer with limited dev ops experience.

More details

 

CircleCI workflow phases

CircleCI pipelines are configured using so-called workflows. Each of them can consist of multiple phases. Our sample Rails pipeline app begins with a setup phase, that is responsible for downloading, installing and caching all the dependencies.

test phase is parallelized to speed up specs execution. Parallelization is not needed for this particular sample app because test suite is small, but mature Rails apps test suites can take even over an hour to execute so parallelizing them is often necessary.

deploy phase pushes the newest code changes to the production server. In this tutorial I describe how to set up Heroku integration.

CircleCI config.yml for Rails apps

Let’s start from a final form of .circleci/config.yml file and down the road I’ll explain it more in detail. This particular config comes from one of my side projects that I’ve recently open sourced. Feel free to check it, and it’s corresponding CircleCI dashboard.

version: 2
jobs:
  setup:
    docker:
      - image: circleci/ruby:2.6.2-node
    steps:
      - checkout

      - run: gem update --system
      - run: gem install bundler

      - restore_cache:
          keys:
            - bundle-{{ checksum "Gemfile.lock" }}
      - run: bundle install --path vendor/bundle
      - save_cache:
          key: bundle-{{ checksum "Gemfile.lock" }}
          paths:
            - vendor/bundle

      - restore_cache:
          keys:
            - yarn-{{ checksum "yarn.lock" }}
      - run: yarn install --cache-folder ~/.cache/yarn
      - save_cache:
          key: yarn-{{ checksum "yarn.lock" }}
          paths:
            - ~/.cache/yarn

      - run: bundle exec rake webpacker:compile
      - save_cache:
          key: webpack-{{ .Revision }}
          paths:
            - /home/circleci/project/public/packs-test/
  test:
    docker:
      - image: circleci/ruby:2.6.2-node
        environment:
          DATABASE_URL: postgresql://postgres:secret@localhost:5432
          REDIS_URL: redis://redis@localhost:6379
      - image: circleci/postgres:11
        environment:
          POSTGRES_USER: postgres
          POSTGRES_DB: price_watcher_test
          POSTGRES_PASSWORD: secret
      - image: circleci/redis:latest
    parallelism: 2
    steps:
      - checkout
      - restore_cache:
          keys:
            - webpack-{{ .Revision }}
      - restore_cache:
          keys:
            - bundle-{{ checksum "Gemfile.lock" }}
      - run: gem update --system
      - run: gem install bundler
      - run: bundle install --path vendor/bundle
      - run: sudo apt install postgresql-client
      - run: dockerize -wait tcp://localhost:5432 -timeout 1m
      - run: bundle exec rake db:create
      - run: bundle exec rake db:schema:load
      - run:
          name: Additional checks
          command: |
              if [ $CIRCLE_NODE_INDEX = 0 ]; then
                bundle exec bundle-audit update
                bundle exec bundle-audit check
              elif [ $CIRCLE_NODE_INDEX = 1 ]; then
                bin/rubocop
              fi
      - run:
          name: Specs
          command: |
            TESTFILES=$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
            bundle exec rspec $TESTFILES --profile 10 --format RspecJunitFormatter --out ~/spec-timings/rspec.xml --format progress
      - store_test_results:
          path: ~/spec-timings
  deploy:
    docker:
      - image: buildpack-deps:trusty
    steps:
      - checkout
      - run:
          name: Deploy to Heroku
          command: |
            cat >~/.netrc <<EOF
            machine api.heroku.com
              login $HEROKU_EMAIL
              password $HEROKU_API_KEY
            machine git.heroku.com
              login $HEROKU_EMAIL
              password $HEROKU_API_KEY
            EOF
            chmod 600 ~/.netrc
            curl https://cli-assets.heroku.com/install.sh | sh
            git push https://heroku:[email protected]/rails-app.git master
            heroku run rake db:migrate -a rails-app
          no_output_timeout: 10m

workflows:
  version: 2
  setup_and_test:
    jobs:
      - setup
      - test:
          requires:
            - setup
            workflows:
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

Docker images based setup

Docker is a first-class citizen in CircleCI workflows setup. You can define the main image inside which tests and checks are executed, as well as supporting images, e.g., for databases.

Thanks to this approach time spent on setting up the environment can be minimized thus speeding up the whole process. CircleCI offers a whole array of supported images. They should be enough for most CI scenarios, but in case your requirements are more complex you can always use your own Docker image published into the public or private repository.

Caching dependencies

CircleCI offers powerful caching features. For our sample Rails app, we need to install both Ruby Gems and Node modules dependencies, as well as results of Webpack assets compilation. Cache keys for dependencies are a checksum of lock files, so an actual install only takes place when something changed. For Webpack compilation we use .Revision variable as a cache key to refresh it with every new commit.

Using this approach allows minimizing the time spent on installing dependencies. It is done only once in a single-threaded job, then multithreaded test jobs can reuse the cache.

Parallel execution of Rails Rspec specs in CircleCI

A test suit of a legacy Rails app can be very slow to run, discouraging developers from using it. Parallelizing specs execution is a no brainer approach to optimizing your CI pipeline speed.

Rails 6 is going to add native support for parallel specs, but CircleCI already offers a decent solution to perform it.

Check out the following config lines:

- run:
  name: Specs
  command: |
    TESTFILES=$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
    bundle exec rspec $TESTFILES --profile 10 --format RspecJunitFormatter --out ~/spec-timings/rspec.xml --format progress
- store_test_results:
  path: ~/spec-timings

It creates a TESTFILES variable, that contains spec file names, split equally for each parallel job. A cool feature is that thanks to saving specs execution speed in ~/spec-timings, each worker runs specs with similar execution time, so there is no danger of a single overworked job delaying the whole suit.

With this config, you can speed up your reduce spec suite time by adding more workers and increasing the parallelism config option.

Execute custom tasks

Additional tasks like Rubocop linting or bundler-audit security checks are an invaluable addition to a robust CI pipeline.

You could add a seperate workflow job for them, but they are usually much faster to execute than the actual specs, so paying for an additional job is not needed.

Instead, you can add executing those check scripts to single jobs from parallelized spec workers using the following config:

- run:
  name: Additional checks
  command: |
    if [ $CIRCLE_NODE_INDEX = 0 ]; then
      bundle exec bundle-audit update
      bundle exec bundle-audit check
    elif [ $CIRCLE_NODE_INDEX = 1 ]; then
      bin/rubocop
    fi

CIRCLE_NODE_INDEX set by CircleCI represents the index of a parallel worker job. With this config first of the parallel workers performs security checks using bundle-audit second lints codebase with Rubocop etc.

Automatic deployments to Heroku

Depending on your infrastructure setup a deployment script will differ. I explain how to setup automatic deployments for Heroku because it is a bit more complex due to Heroku CLI login hack needed:

  deploy:
    docker:
      - image: buildpack-deps:trusty
    steps:
      - checkout
      - run:
          name: Deploy to Heroku
          command: |
            cat >~/.netrc <<EOF
            machine api.heroku.com
              login $HEROKU_EMAIL
              password $HEROKU_API_KEY
            machine git.heroku.com
              login $HEROKU_EMAIL
              password $HEROKU_API_KEY
            EOF
            chmod 600 ~/.netrc
            curl https://cli-assets.heroku.com/install.sh | sh
            git push https://heroku:[email protected]/rails-app master
            heroku run rake db:migrate -a rails-app
          no_output_timeout: 10m
  ...

workflows:
  version: 2
  setup_and_test:
    jobs:
      - setup
      - test:
          requires:
            - setup
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: master

This config assumes that your Heroku app is called rails-app so make sure to change it accordingly. You must add HEROKU_API_KEY and HEROKU_EMAIL variables via CircleCI UI. HEROKU_API_KEY is generated by running:

heroku authorizations:create

Workflows config ensures that deploy workflow phase executes only for a master branch pushes. You might want to tweak this config if your GitHub deployment workflow is different.

Modifying .netrc file is necessary if you want to execute migration or any other rake tasks after each deploy using authenticated Heroku CLI.

GitHub integration settings

How to set up a productive GitHub workflow is a story for another blog post. Just make sure to select building CI pipeline only for pull requests and cancel redundant build in CircleCI options. Otherwise, your CI queue might get stuck if there are more developers pushing commits to the same project.

Summary

I hope you’ll find some of those tips useful when setting up your continuous integration pipeline using CircleCI. Let me know if you notice some ways how this setup could be improved.