Upgrading Kamal from v1.4 to v1.5

This isn’t an extensive upgrade guide by any means but I wanted to highlight two of the most common stumbling blocks that came with the v1.5 release. They’re changes for the better but can potentially break your build process, it’s an easy adjustment though.

#732 - Always send the clear env to the container

With Kamal < v1.4 your clear environment variables would only get updated when you ran a kamal env push which updated the related .env files on your server. With v1.5 the clear environment variables are set with every docker run(deploy) command, you’ll see them in the docker run output. For instance, if you have your clear environment variables set to something like this:

servers:
  web:
    hosts: 867.53.0.9
    env:
      clear:
        WEB: true
        DB_POOL: 10
        RAILS_MAX_THREADS: 10
        APPSIGNAL_HOSTNAME: web

It maps to something like this in the docker run commands:

docker run --detach --restart unless-stopped \
--name hey-web-b9b6e3aa4c2aac7dc7f3b8847179bc446b8b97f9 \
-e KAMAL_CONTAINER_NAME="hey-web-b9b6e3aa4c2aac7dc7f3b8847179bc446b8b97f9" \
-e KAMAL_VERSION="b9b6e3aa4c2aac7dc7f3b8847179bc446b8b97f9" \
--env-file .kamal/env/roles/hey-web.env \
--env RAILS_ENV="staging" \
--env WEB="true" \
--env DB_POOL="10" \
--env RAILS_MAX_THREADS="10" \
--env APPSIGNAL_HOSTNAME="web"
[truncated]

Previously with v1.4 everything was loaded with just the --env-file option and pointing at the specific .env file for that role/server/accessory. With v1.5 secrets are still loaded with the --env-file option and then for each clear variable it’s passed in as --env as-is.

This is a great change because now you know that if a deploy went out, it picked up the current clear environment variables from your deploy.yml. Secret and clear variables also typically live in different places. Clear variables are meant for things like how many Sidekiq processes to spin up, the app environment, setting the database connection pool size, etc. They’re also stored in plain text in your git repository for all to see with the current values. Secrets on the other hand might be in Rails credentials, 1Password, or any other secret store. Secrets don’t usually change as often and it’s often a very coordinated change such as rotating out an API key.

#700 - Build from a git archive

With v1.4 when Kamal goes to build your container it utilizes the current state of files on your machine. This is great when you’re getting started with Kamal as you can make a change and then rebuild without having to commit each change or tweak to git. The downside is as you’re building your container you might accidentally include secrets in your container or large artifacts that are just part of your local environment but shouldn’t be in the final image. This is resolved of course by ensuring your .dockerignore file is properly excluding anything that doesn’t need to be part of the final build. Sidenote: I highly recommend poking around your built container to remove anything that's not essential to your app running, du -ah | sort -h is usually a great starting point.

This PR changes the default behavior for this to favor loading the Docker build context based on a git archive of HEAD within your repository. This way it’s always going to build the same container for that SHA since it’s just an archive of that specific SHA. This is a nice step forward in consistency but can be a bit confusing when your build stops working, it’s most likely because it’s missing a file required for your build that was included from your local working directory previously.

The old behavior is still available and pretty simple to enable by specifying the build context within Kamal. This changes the build process back to utilizing the current directory for the context instead of the git archive.

builder:
  context: "."

There’s also a PR(#798) that was just merged in that’ll supersede this behavior to utilize a git clone instead to help resolve issues that bubbled up from the git archive approach.

MRSK, I mean Kamal is still pretty young, v0.0.1 was released just over a year ago. Some of this behavior is bound to change as more people utilize a tool like this and the pace of improvements has only continued to pick up, it's great to see the adoption.