In theory, you can run both Rails web server and Sidekiq process on one 512mb Heroku dyno. For side projects with small traffic, saving $7/month always comes in handy. Unfortunately when trying to fit two Ruby processes on one dyno you can run into memory issues and leaks. In this post, I will explain how you can reduce memory usage in Rails apps.
Recently, I read a great article by Bilal Budhani explaining how to run Sidekiq process alongside Puma on one Heroku dyno. After applying it to one of my side projects, I started running into those dreaded R14 errors.
Memory usage spiked followed by a bunch of memory errors and automatic restart
I did some digging using Scout and, after a couple of optimizations memory usage charts started looking like this:
Stable memory usage followed by garbage collection
Here’s what I did:
Put your Gemfile on a diet
There is a Gem for that… In the Ruby world, dropping in a Gem to solve a problem is usually the “easiest” solution. Apart from other costs, memory bloat is one that can easily get overlooked.
The best way to check how much memory each of your gems consumes is to use derailed benchmarks. Just add:
Gemfile and run
bundle exec derailed bundle:mem.
My project powers Twitter bot profile. I was surprised to find out that a popular twitter gem uses over
13 MB of memory on startup. First I replaced it with its lightweight alternative grackle (
~1 MB) and finally ended up writing a custom code making a HTTP call to Twitter API. I also managed to get rid of koala gem (
~2 MB) in the same way.
Another quick win was replacing gon gem (
Use jemalloc to reduce Rails memory usage
jemalloc is an alternative to official MRI memory allocator. On Heroku, you can add jemalloc using a buildpack. For my app, it resulted in ~20% memory usage decrease. Just make sure to test your app with jemalloc thoroughly on a staging environment before deploying it to production.
You can also check out my other blogpost for tips on how to use Ruby with jemalloc for Docker and Dokku based Rails apps.
Limit concurrency and workers
For a side project with limited traffic you probably don’t need a lot of throughput anyway. You can limit memory usage by reducing Sidekiq and Puma workers and threads count. Here’s my
Although we specify 1 as the max number of threads, Puma can spawn up to 7 threads. With those minimal settings, Smart Wishlist is still able to process around 100k Sidekiq jobs daily and serve both React frontend and mobile JSON API.
Optimize JSON parsing
All those Sidekiq jobs are necessary to download up to date prices from iTunes API (requests batching is on my TODO list) and send push notifications about discounts. This means there’s quite a lot of JSON parsing going on there. There is an simple fix which can help optimize both memory usage and performance in such cases:
oj gem offers a Rails Compatibility API, which hooks into
JSON.parse calls, improving their performance and memory usage.
Working in constrained environments is a great way to flex your programming muscles and discover new optimization techniques. In theory, you could always solve memory issues by throwing more money at your servers, but why not keep those $7 instead?
Check out my other blog post for more tips on how to improve performance and reduce memory usage of Rails apps.