asdf and Docker for Managing Local Development Dependencies

Have you ever updated a database for one project just to discover that you accidentally broke another? Ensuring the isolation between your local dependencies can save you hours of rolling back a breaking update or resolving intertwined dependencies. In this blog post, I describe how you can manage your local development stacks using Docker containers and asdf.

What are the dangers of using globally installed dependencies?

Before, I’ve used Homebrew to manage my local dependencies, e.g., PostgreSQL database. Many tutorials recommend it as a quick & easy way to install packages on MacOS. One critical issue of this approach is that it does not permit the installation of different versions at the same time. Also, specifying a granular minor version is not straightforward.

I’ve finally decided to ditch using Homebrew for managing development dependencies after the update of the Vim package somehow bumped my local PostgreSQL major version… It resulted in cumbersome data restoration and a couple of wasted hours.

While conducting my Rails performance audits, I sometimes see projects not paying attention to the consistency of their local and production dependency versions. I understand that it’s not always easy to match the minor release of PostgreSQL or Redis ideally. But the mismatch can result in an unexpected downtime if some obscure feature is incompatible between releases.

Let’s discuss how we can improve this setup.

Docker containers for local dependencies

Docker let’s you run the processes for each of your dependencies in a separate container. With so-called Docker Compose you can describe your stack in an easy to understand yml file. Let’s see a sample configuration for PostgreSQL and Redis, a standard Ruby on Rails setup:

.env.development.database

POSTGRES_USER=project1_postgres
POSTGRES_PASSWORD=project1_password
POSTGRES_DB=rails_project1_db

docker-compose.yml

services:
  postgres:
    image: postgres:12.2-alpine
    env_file:
      - .env.development.database
    ports:
      - '5432:5432'
    volumes:
      - project1_pg_data:/var/lib/postgresql/data
  redis:
    image: redis:6.2.5-alpine
    ports:
      - '6379:6379'

volumes:
  project1_pg_data:

Notice that we don’t dockerize the Rails app itself, only the dependencies. I’m not a proponent of dockerizing the apps for local development because it causes a cumbersome indirection.

As you can see, we can fine-tune the exact versions of our dependencies and their ports. Data is persisted using a Docker volume.

To spin up your stack run docker compose up command. A new PostgreSQL database will be created during the initial launch based on the configuration from the .env.development.database file. To stop the processes, you have to run docker compose stop. Both Redis and PostgreSQL are exposed on their standard ports.

This approach is very flexible. I use it to run the test suit of my rails-pg-extras library against PostgreSQL versions 11, 12, and 13 in a single command. Each PG version is running on a different port. You can check out the repository for implementation details.

I usually don’t keep the docker-compose.yml file in the repository. A good practice is to keep only a docker-compose.yml.sample in a version control, so that every developer can decide on the setup of his local environment.

Advert

My RDS migration guide is now out!

This is the eBook that I wish existed when I was first tasked with moving the Heroku database to AWS as a developer with limited dev ops experience.

More details

 

Configure asdf for managing executables

Let’s now discuss how to work with different versions of executable dependencies like Ruby or NodeJS. I’ve been using various version managers, i.e., RVM, NVM, rbenv… Recently, I’ve discovered “the one to rule them all”, namely asdf.

asdf lets you specify executable versions using a simple config file. You can install it by running:

git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.8.1

and adding

. $HOME/.asdf/asdf.sh

to your .bashrc or .zshrc file.

After installing asdf, you have to add plugins for executables that you want to use:

asdf plugin-add ruby https://github.com/asdf-vm/asdf-ruby.git
asdf plugin-add nodejs
asdf plugin-add yarn

and run asdf install to install dependencies specified in .tool-versions file.

Here’s a sample config file for one of my Rails projects:

ruby 2.7.3
nodejs 14.17.1
yarn 1.22.10

There’s not much to explain. After entering a folder with this file, all the executables will be switched to their specified versions. You can define default versions using a .tool-versions file in your home directory. There are dozens of asdf plugins available for different programming languages and command line tools.

Presence of this file does not mean that everyone from your team must start using asdf. .tool-versions file checked into the repository can work as a single source of truth on which versions should be used, regardless of how you do it.

Summary

I hope that the above tips will help you simplify your local development setup. Before I’ve started using this approach, I’ve wasted dozens of hours resolving dependencies hell. I even avoided updates out of the fear of breaking something in my other projects. asdf together with Docker containers makes switching between projects using different tech stacks virtually seamless.