Embracing Kamal secrets

Kamal 2 ditched .env in favor of .kamal/secrets which is great because it isolates Kamal-specific configuration from your application configuration. Kamal 2 also removed the need to run kamal env push when your secrets change and instead your secrets are now pushed with each deploy.

There are two main approaches you can utilize with Kamal's secrets.

1. Store secrets as-is in plain text in .kamal/secrets. You might've been doing this if you were using .env previously.
2. Store secrets in a password vault(1Password, LastPass, Bitwarden, AWS Secrets Manager, Doppler). You're probably using a password manager/vault for other credentials, Kamal now makes it dead simple to utilize an existing vault to fetch credentials when you deploy.

Storing secrets in plain text

RAILS_MASTER_KEY=this-is-the-actual-key
KAMAL_REGISTRY_PASSWORD=yep-the-actual-password

The plain text approach has a few downsides:

1. You're storing the credentials in plain text on your machine.
2. If multiple people are working on the same app you have to tell everyone to update any credential when it changes. This becomes more painful because Kamal is now pushing up secrets with each deploy. For example, you could have someone deploy with the old version of the credential which might result in a failed deploy or revert a necessary configuration change.
3. It's not as obvious what should exist in .kamal/secrets because you should not commit this file to git.
4. If you need to rotate out your container registry(docker hub) token because of expiration or exposure and you're working on multiple applications you'll now need to update this in each application.

The upside for using plain text is it's simple to manage for small solo projects and you can verify the contents by just opening the file. There's also no need to install the CLI tool for your given password manager.

Storing secrets in a vault

PERSONAL_SECRETS=$(kamal secrets fetch --adapter 1password --account my.1password.com --from VaultName/kamal-secrets KAMAL_REGISTRY_PASSWORD)

APP_SECRETS=$(kamal secrets fetch --adapter 1password --account my.1password.com --from AppVault/app-name RAILS_MASTER_KEY)

KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract KAMAL_REGISTRY_PASSWORD $PERSONAL_SECRETS)

RAILS_MASTER_KEY=$(kamal secrets extract RAILS_MASTER_KEY $APP_SECRETS)

Kamal 2 introduced a few new secrets commands to extract secrets from various password vaults. You essentially fetch an item from your password vault and then extract each secret as needed from that item.

There's quite a few upsides to utilizing a password vault for secrets:

1. You can share a password vault/entry that contains everything needed to deploy that specific application.
2. If a credential is updated, you update it in your vault, and your deploys automatically fetch the latest value for anyone deploying.
3. You can control access to application-specific credentials at the vault level instead of some other out-of-band process.
4. There's no more copying/pasting of credentials from wherever into the secrets file.

Inspecting secrets

When you're getting started configuring the secrets for an application, it's also helpful to see what Kamal is seeing for the values of your secrets. You can easily see these values by running kamal secrets print. This helps to ensure that fetching secrets from your vault is working correctly and setting the secret as expected.

The various Kamal secret files

Kamal checks in a few places for secrets, depending on how you're deploying.

1. .kamal/secrets - Ideally, you can put all of your secret lookups in this one file. If you're using destinations, you can also utilize KAMAL_DESTINATION in here to conditionally load from the correct vault, staging vs. production, for example.
2. .kamal/secrets-common - If you're using destinations you'd put your common secrets across destinations in here. For instance, maybe your KAMAL_REGISTRY_PASSWORD or BUNDLE_GITHUB__COM.
3. .kamal/secrets.destination-name - If you're using destinations(environments) with Kamal you'll want to put your destination-specific secrets in this file.

One approach that's been useful so you don't have to maintain a secrets.staging and secrets.production file is to load the correct entry from your vault based on the current destination. For example, you can lookup your secrets by interpolating the destination when you fetch your secrets.

DESTINATION_SECRETS=$(kamal secrets fetch --adapter 1password --account my.1password.com --from Vault/app-$KAMAL_DESTINATION RAILS_MASTER_KEY)

If you're able to organize and namespace your secrets by destination then a simple .kamal/secrets file should do the trick for most applications.

Specifying the secrets for a role or accessory

When you deploy with Kamal it creates a file on-the-fly based on the values from your vault but also specifically what's required for that role or accessory. This file is then referenced with the docker run command with a file such as --env-file .kamal/apps/app-name/env/accessories/db.env.

For example, just because you have RAILS_MASTER_KEY in .kamal/secrets does not mean it'll be available for all roles and accessories for your deploy. You need to specifically list out the secrets that each role and accessory needs.

This is intentional and by design so that you can expose only the necessary secrets for that role or accessory. For example, a database accessory should not need RAILS_MASTER_KEY but it probably needs your DB_PASSWORD.

servers:
  web:
    env:
      secret:
        - RAILS_MASTER_KEY
        
accessories:
  db:
    env:
      secret:
        - DB_PASSWORD

There's ongoing work to add a few more password managers as well such as GCP's secret manager and Enpass. You can also open a pull request if your password manager isn't currently available as long as you can fetch secrets from a CLI tool.

Versions

• Kamal - 2.4.0