Hosting multiple Postgres databases with Kamal

When deploying with Kamal and using accessories such as Postgres, sometimes you'll want to run a few Postgres servers/containers/accessories. Maybe on different servers or on the same server from the same deploy recipe. One of the downsides of trying to do that with Kamal is that when you're booting these containers, they're all relying on the same environment variables, aka POSTGRES_PASSWORD.

For example, say you want to run a primary and secondary Postgres server. One that's optimized for reading and the other for writing. Or maybe you want to run isolated Postgres servers for each of your services or customers, but you're deploying them from the same repository. To configure these, you'll need to set a secret in Kamal for POSTGRES_PASSWORD.

accessories:
  postgres:
    env:
      secret:
        - POSTGRES_PASSWORD
 

You could share the password for both databases, but that's not really ideal, especially if you need to ensure unique credentials for each database. There's not currently an easy way to remap or rename a secret with Kamal, but with the postgres image it's fairly easy to do with the POSTGRES_PASSWORD_FILE option. The Postgres image supports a handful of configuration options that can be set by passing a file, and POSTGRES_PASSWORD happens to be one of them.

Kamal actually has a pretty simple approach to solving this because of the support for using a file with POSTGRES_PASSWORD_FILE which is through the files mapping for each accessory.

If you look at the docs for accessories, you'll see that within the Copying Files section it talks about how the files will be evaluated as ERB files, great. Since they're evaluated as ERB, we can fetch our POSTGRES_PASSWORD values from wherever we need, whether that's an existing Kamal secret, a different password vault, or just plain text. Once we have our passwords in those files we can just add it to the files mapping and our accessory will boot with with them.

We're going to boot Postgres 17.2 with the goal of having a primary and secondary Postgres server.

I've already provisioned a server, so I'm just going to focus on the accessories portion to get our two Postgres accessories up and running. We're starting off with a pretty basic deploy recipe and we'll just add the accessories from here.

service: hotdonuts

image: nickhammond/hotdonuts

servers:
  web:
    hosts:
      - 64.23.193.32

registry:
  username: nickhammond
  password:
    - KAMAL_REGISTRY_PASSWORD

builder:
  arch: amd64
  pack:
    builder: "heroku/builder:24"
    buildpacks:
      - heroku/ruby
      - heroku/procfile

We can go ahead and start working on our primary Postgres accessory, which will still utilize kamal secrets to fetch our POSTGRES_PASSWORD. First, lets put our POSTGRES_PASSWORD in a file so that we can upload it for use in our accessory container. Assuming you already have POSTGRES_PASSWORD configured within your Kamal secrets we can go ahead and fetch it from there and pull out just the password. The contents of this file will be run through ERB so we can utilize ERB to fetch the secret and extract just what we need, which is just the value of the password.

<%= `kamal secrets print | grep POSTGRES_PASSWORD`.split("=").last.strip %>

Go ahead and save this to pg-password-primary.txt.erb on your local machine. Since we're fetching from kamal secrets this file is also fine to commit to your repository. This file could also be stored in .kamal/pg-password-primary.text.erb or somewhere else, up to you.

With that file in place, we can go ahead and add the file mapping as well as point the Postgres container toward the file for the POSTGRES_PASSWORD value.

service: hotdonuts

image: nickhammond/hotdonuts

servers:
  web:
    hosts:
      - 64.23.193.32

accessories:
  postgres:
    image: postgres:17.2
    roles:
      - web
    env:
      clear:
        POSTGRES_USER: "primary"
        POSTGRES_PASSWORD_FILE: "/pg-password-primary"
    files:
      - pg-password-primary.txt.erb:/pg-password-primary
    directories:
      - data:/var/lib/postgresql/data

Notice that we don't have a secret configured for POSTGRES_PASSWORD, just a clear value that points to the file.

POSTGRES_PASSWORD_FILE: "/pg-password-primary"

And then we're uploading it into the container to its final destination at /pg-password-primary, this could be anywhere as long as the two paths match.

files:
  - pg-password-primary.txt.erb:/pg-password-primary

With our primary accessory configured, we can go ahead and add in the secondary Postgres accessory. Go ahead and create the POSTGRES_PASSWORD file for the secondary accessory at pg-password-secondary.txt.erb. I'm just going to do this one as plain text but this could also live in kamal secrets. Since this is in plain text, you don't want to commit this to git. Also, the .txt.erb extension isn't necessary.

secret

The location of these files is also up to you, these probably belong within the .kamal directory, but I ended up just putting them here for this post.

With that file in place, we can go ahead and add our secondary Postgres server and utilize pg-password-secondary.txt.erb for our POSTGRES_PASSWORD_FILE.

service: hotdonuts

image: nickhammond/hotdonuts

servers:
  web:
    hosts:
      - 64.23.193.32

accessories:
  postgres:
    image: postgres:17.2
    roles:
      - web
    env:
      clear:
        POSTGRES_USER: "primary"
        POSTGRES_PASSWORD_FILE: "/pg-password-primary"
    files:
      - pg-password-primary.txt.erb:/pg-password-primary
    directories:
      - data:/var/lib/postgresql/data

  postgres_secondary:
    image: postgres:15
    roles:
      - web
    env:
      clear:
        POSTGRES_USER: "secondary"
        POSTGRES_PASSWORD_FILE: "/pg-password-secondary"
    files:
      - pg-password-secondary.txt.erb:/pg-password-secondary
    directories:
      - data:/var/lib/postgresql/data

registry:
  username: nickhammond
  password:
    - KAMAL_REGISTRY_PASSWORD

builder:
  context: "."
  arch: amd64
  pack:
    builder: "heroku/builder:24"
    buildpacks:
      - heroku/ruby
      - heroku/procfile

With those file mappings in place and the clear POSTGRES_PASSWORD_FILE values set we no longer need to set POSTGRES_PASSWORD for either of our accessories.

Now that we have both of our accessories configured in our deploy recipe we can go ahead and boot them.

When we boot the accessories with kamal accessory boot all, you'll see a few things happening.

1. It uploads a password file to /pg-password-primary and /pg-password-secondary
2. When it boots each accessory, you'll now see it mounting the file as well as pointing to the file with an --env flag --env POSTGRES_PASSWORD_FILE="/pg-password-primary" for our primary and --env POSTGRES_PASSWORD_FILE="/pg-password-secondary" for our secondary accessory.

Once those boot, we can verify that our accessories are running by checking the kamal details output:

You can see we have a status of "Up 18/21 seconds" for both of our accessories, yay! If there was an issue with our containers booting the status would be a constant restart cycle and you'd see "restarting..." under the status.