Deploying a Rails app with Kamal, Heroku-style
I’ve been really enjoying working with Kamal as my deployment tool of choice, so much so that I’ve also been helping others migrate their setup to Kamal. It’s a refreshing take on deploying your containerized app that reminds me of the simplicity of the Capistrano days.
The Docker build process is incredibly powerful but also incredibly customizable which takes away one of my favorite principles of a tool, convention over configuration. The build process of your app is not what’s adding the value to your product/application/company at the end of the day. If you’re spending time ensuring that your apt(node, bundler, etc) installs are always cached it’s time spent not fixing or adding a real feature. Or ensuring that you purge out all of the cache files from your install steps so that you have the smallest possible container which helps ensure faster deploys but ultimately isn't contributing to your product's success. Sure, there’s a recommended line to copy/paste from a previous app that you’ve worked on or something that you see everywhere, something like this:
RUN --mount=type=cache,id=dev-apt-cache,sharing=locked,target=/var/cache/apt \
--mount=type=cache,id=dev-apt-lib,sharing=locked,target=/var/lib/apt \
apt-get update -qq && \
apt-get install -y nodejs build-essential default-libmysqlclient-dev \
git libvips pkg-config libmagickwand-dev && \
rm -rf /var/lib/apt/lists /var/cache/apt/archives
This is a line in a production Dockerfile to install the necessary apt-packages for Bundler to be able to install the gems for an app along with some image processing tooling(libvps, libmagickwand-dev). We’re also mounting a cache directory so that our build runs faster but then we’re also ensuring that the apt archives are purged out to prune our image.
I have this line sprinkled in a bunch of apps that are container-based. It works, it’s fast, it’s essential, and maybe there could be some more optimization to it but that’s not the point. The point is, it’s brittle. It’s the equivalent of still writing manual bash scripts when you have tools like Chef, Puppet, Ansible, etc. that are much better at standardizing those types of tasks.
What if we don’t have to write a Dockerfile but can still containerize our application into a Docker bootable image for Kamal? Here’s where Buildpacks come in.
Buildpacks
If you’ve ever deployed on Heroku, you’ll know that you didn’t have to write a Dockerfile. Essentially all you needed to do was follow the recommended framework, app, language, documented way and Heroku will just figure it out and make it work. They’re helping you follow the convention over configuration for that framework/language essentially, you’re not even doing anything Heroku-specific(just a Procfile) but just specific to the tools that you’re already using.
If it’s a Rails app, you need a Gemfile. If it’s a node app, you need a package.json file. Buildpacks work in a few phases but the main two phases that are important to understand are:
- Detect - When the buildpack runs this is the phase that looks for various version or necessary runtime files. Files like your Gemfile or your package.json file. If it finds one of these files that means that it will trigger(pass) a build in one of the upcoming phases. If it doesn’t detect(fail) anything that it has a buildpack for then nothing will happen in the build phase.
- Build - If a detection passed for a specific buildpack(node.js, Ruby), this is where the build process kicks in. Based on each buildpack that passed detection, the build process will run the build process for that specific buildpack. For example, it’s not going to install packages for PHP’s composer buildpack if it didn’t find a composer.json file in your app. But if it detected Ruby because of the presence of a Gemfile it’s going to run a
bundle install.
Heroku has been dealing with this build/deploy process since 2007(17 years ago) and it’s only gotten better and better. Better because of customer feedback complaining about why their app didn’t build for a specific version of node. Better because their official buildpacks have been open source for a good chunk of that time, the Ruby buildpack for instance since 2011?(first issue on GitHub).
You also don’t have to do it all the official Heroku way, there are 3rd party buildpacks and you can even build and easily use your own. For example, need ffmpeg to do some video processing? There’s a buildpack for that. Need to convert HTML to PDF with wkhtmltopdf? There’s a buildpack for that.
What’s exciting about this is that Heroku lovingly open-sourced their Cloud Native buildpacks earlier this year. The Cloud Native Buildpacks project provides a spec and tooling to pack your app into an image, from the website:
The CNB project maintains a set of specifications and tooling that make it possible to write buildpacks and orchestrate them together in a build system. Additionally, community providers such as Google, Heroku, and the Paketo project maintain buildpacks for a wide variety of language families, and you can discover even more by searching the buildpack registry.
The Cloud Native Buildpacks project is also a Cloud Native Computing Foundation incubating project which is part of the Linux Foundation.
Anyways, let’s build an app, the Heroku way.
Hot Donuts
I like donuts, they’re a nice treat. They’re also great when they’re hot and fresh, introducing the Hot Donuts app. It’s an app for the local donut shop to let the people know that fresh donuts are up.
More importantly, it’s a single-file Rails app that uses Sqlite and we’re going to deploy it without writing a Dockerfile via Kamal.
First, since this will be a Rails app and we’ll be using Bundler we’ll need a Gemfile.
Gemfile
source "https://rubygems.org"
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gem "kamal", github: "nickhammond/kamal", branch: "buildpacks"
gem 'rails'
gem 'sqlite3'
Normally you wouldn’t need to specify Kamal in your Gemfile but we’re going to be deploying via bundle exec so that we can get Buildpack support from my branch for Kamal.
Here’s our full Rails app, save this in app.rb. It’s worth a simple skim through to see what’s going on but it just does a couple of things:
- Has an /up healthcheck endpoint for Kamal.
- The homepage lets the user know if hot donuts are available.
- If donuts are not available then it lets the user put in a request for a donut. As people request donuts then we just increment a counter to show the demand for a fresh batch of donuts.
- We also just have a simple http basic auth endpoint so that only the donut shop owner can flip the switch to let people know donuts are available.
app.rb
# frozen_string_literal: true
require 'bundler/setup'
require "rails"
require "active_record/railtie"
require "rails/command"
require "rails/commands/server/server_command"
Rails.logger = Logger.new(STDOUT)
database = "#{Rails.env}.sqlite3"
ENV['DATABASE_URL'] = "sqlite3:#{database}"
ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: database)
ActiveRecord::Base.logger = Logger.new(STDOUT)
ActiveRecord::Schema.define do
create_table :donuts, force: true do |t|
t.boolean :available
t.bigint :orders, default: 0
t.timestamps
end
end
class App < ::Rails::Application
config.consider_all_requests_local = false
config.eager_load = true
config.secret_key_base = 'secret'
routes.append do
get "up" => "rails/health#show", as: :rails_health_check
root 'donuts#index'
put 'donuts' => 'donuts#update', as: :donuts
put 'order' => 'donuts#order', as: :order
end
end
class Donut < ActiveRecord::Base
end
Donut.create!(available: true) if Donut.count.zero?
class DonutsController < ActionController::Base
include Rails.application.routes.url_helpers
http_basic_authenticate_with name: "hot", password: "donuts", only: :update
before_action :load_donut
def index
render inline: """
<!DOCTYPE html>
<html>
<head>
<title>Hot donuts</title>
<meta name='viewport' content='width=device-width,initial-scale=1'>
<%= csrf_meta_tags %>
<%= stylesheet_link_tag 'https://cdn.simplecss.org/simple.min.css' %>
</head>
<body>
<main>
<% if notice %>
<p><mark><%= notice %></mark></p>
<% end %>
<p class='notice'>
<%= content_tag(@donut.available? ? :mark : :span) do %>
Hot Donuts are <%= @donut.available? ? 'available!' : '<strong>not available.</strong'.html_safe %>
<% end %>
<% unless @donut.available? %>
<p>
<% if @donut.orders > 0 %>
<em><%= pluralize(@donut.orders, 'person') %> ordered a hot donut.</em>
<% else %>
Be the first to order a hot donut!
<% end %>
</p>
<%= form_with url: order_path, method: :put do |form| %>
<%= form.submit 'Order a donut' %>
<% end %>
<% end %>
</p>
<%= form_with url: donuts_path, method: :put do |form| %>
<%= form.submit @donut.available? ? 'Sold out' : 'Make available' %>
<% end %>
</main>
</body>
</html>
"""
end
def update
@donut.toggle!(:available)
redirect_to root_path
end
def order
@donut.increment!(:orders)
redirect_to root_path, notice: "Thanks for ordering a donut!"
end
def load_donut
@donut = Donut.last
end
end
App.initialize!
Rails::Server.new(app: App, Host: "0.0.0.0", Port: 80).start
Lastly, we just need a Procfile which will specify the default web command that the container will boot with:
Procfile
web: ruby app.ru
Kamal
With our app setup locally we can start configuring the Kamal portion to get our app deployed.
The server: I’ve already created a server in Digital Ocean with the ubuntu-24-04-x64 image that has my SSH key automatically added to the server so we just need to make sure we point to the IP address. Here’s the curl request details for creating the machine via the Digital Ocean API:
curl -X POST -H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$TOKEN'' \
-d '{"name":"hotdonuts",
"size":"s-1vcpu-2gb",
"region":"sfo3",
"image":"ubuntu-24-04-x64",
"vpc_uuid":"vpc_uuid"}' \
"https://api.digitalocean.com/v2/droplets"
Normally you’d do a kamal init but we’ll just create the essential files with the content we need instead.
In your .kamal/secrets(Kamal 2.0) file you’ll need your Docker API key as a password for Kamal:
.kamal/secrets
KAMAL_REGISTRY_PASSWORD=dckr_pat_x
And then we need our config/deploy.yml which is the default deploy recipe that Kamal looks for.
config/deploy.yml
service: hotdonuts
image: nickhammond/hotdonuts
servers:
web:
hosts:
- 123.456.78.9
registry:
username: nickhammond
password:
- KAMAL_REGISTRY_PASSWORD
builder:
arch: amd64
context: "."
pack:
builder: "heroku/builder:24"
buildpacks:
- heroku/ruby
- heroku/procfile
For our builder config we’re using a few settings specific to buildpacks, the builder structure is also changing with Kamal 2.0 so this might change a bit.
builder:
arch: amd64
context: "."
pack:
builder: "heroku/builder:24"
buildpacks:
- heroku/ruby
- heroku/procfile
Here are some details about our builder config:
- We’re building for amd64 only which is our production environment
- I’ve added the context setting so that we can build from our local context and not have to deal with git yet. Later on you would remove the context and let kamal build your app from the git archive.
builderWith the Pack CLI you can use various builders, we want to ensure that we use the Heroku builder though.buildpacks- We’re setting our buildpacks here so that we include the correct ones and ensure they are added to the detect step.
All of these settings can also be overridden in a project.toml file which is pack’s project descriptor file. We’re going to add one of these because pack doesn’t use a .dockerignore file at the moment and we want to make sure our .kamal/secrets file doesn’t end up in our final image.
project.toml
[_]
schema-version = "0.2"
id = "io.buildpacks.hotdonuts"
name = "Hot Donuts"
version = "0.0.1"
[io.buildpacks]
exclude = [
".kamal/secrets"
]
I think respecting a .dockerignore or .gitignore should somehow be supported with the Pack CLI, maybe it’s possible and I’m missing something but I’ve brought it up as an issue on the Pack CLI repo.
With that in place, we can install the Pack CLI and then deploy our app with Kamal. I’m using Homebrew so you can just install it with:
brew install buildpacks/tap/pack
With that installed we can deploy our app via Kamal!
The first time you deploy with Kamal you’ll need to run the setup command which installs Docker if it isn’t available, pushes your .kamal/secrets file(it will be empty for this app), and a few other things.
bundle install
bundle exec kamal setup
Make sure you use bundle exec so that we can use the new pack builder option.
Yay! Our application is live.
With our app live we should go ahead and deploy it again to see how it utilizes the build cache automatically.
DEBUG [9195f834] ===> BUILDING
DEBUG [9195f834] [builder]
DEBUG [9195f834] [builder] # Heroku Ruby Buildpack
DEBUG [9195f834] [builder]
DEBUG [9195f834] [builder] - Metrics agent
DEBUG [9195f834] [builder] - Skipping install (`barnes` gem not found)
DEBUG [9195f834] [builder] - Ruby version `3.1.3` from `default`
DEBUG [9195f834] [builder] - Using cached Ruby version
DEBUG [9195f834] [builder] - Bundler version `2.5.9` from `Gemfile.lock`
DEBUG [9195f834] [builder] - Using cached version
DEBUG [9195f834] [builder] - Bundle install
DEBUG [9195f834] [builder] - Loading cached gems
DEBUG [9195f834] [builder] - Skipping `bundle install` (no changes found in /workspace/Gemfile, /workspace/Gemfile.lock, or user configured environment variables)
DEBUG [9195f834] [builder] - ! HELP To force run `bundle install` set `HEROKU_SKIP_BUNDLE_DIGEST=1`
DEBUG [9195f834] [builder] - Setting default processes
DEBUG [9195f834] [builder] - Running `bundle list` ... (0.266s)
DEBUG [9195f834] [builder] - Detected rails app (`rails` gem found)
DEBUG [9195f834] [builder] - Rake assets install
DEBUG [9195f834] [builder] - Skipping rake tasks (no `Rakefile`)
DEBUG [9195f834] [builder] - ! HELP Add `Rakefile` to your project to enable
DEBUG [9195f834] [builder] - Done (finished in 0.354s)
Notice that Bundler doesn’t need to download our gems again and also the “Loading cached gems” and “Skipping ‘bundle install’” lines. Another thing to note is we didn't write a Dockerfile, the build related details are just a few lines in our deploy recipe.
But, I don't use Rails.
No worries, here are a few hello world demo apps that you can deploy with Kamal that'll give you the same idea.
nickhammondnickhammondnickhammondnickhammondResources
Big thank you to Schneems for sparking this initial idea but also the whole Heroku team. There's also some great information in this post around how Packs work.
Versions
Kamal:
https://github.com/nickhammond/kamal/commit/85a5a09aacafb852e9905c360a578388ce2ad32d
$ pack version
0.35.0+git-f6b450f.build-6065
$ docker version
Client: Docker Engine - Community
Version: 27.0.3
API version: 1.46
Go version: go1.22.4
Git commit: 7d4bcd863a
Built: Fri Jun 28 14:56:30 2024
OS/Arch: darwin/arm64
Context: desktop-linux
Server: Docker Desktop 4.33.0 (160616)
Engine:
Version: 27.1.1
API version: 1.46 (minimum version 1.24)
Go version: go1.21.12
Git commit: cc13f95
Built: Tue Jul 23 19:57:14 2024
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.7.19
GitCommit: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
runc:
Version: 1.7.19
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0