KMS Encrypted

Simple, secure key management for Lockbox and attr_encrypted

With KMS Encrypted:

Master encryption keys are not on application servers
Encrypt and decrypt permissions can be granted separately
There’s an immutable audit log of all activity
Decryption can be disabled if an attack is detected
It’s easy to rotate keys

Supports AWS KMS, Google Cloud KMS, and Vault

Check out this post for more info on securing sensitive data with Rails

How It Works

This approach uses a key management service (KMS) to manage encryption keys and Lockbox / attr_encrypted to do the encryption.

To encrypt an attribute, we first generate a data key and encrypt it with the KMS. This is known as envelope encryption. We pass the unencrypted version to the encryption library and store the encrypted version in the encrypted_kms_key column. For each record, we generate a different data key.

To decrypt an attribute, we first decrypt the data key with the KMS. Once we have the decrypted key, we pass it to the encryption library to decrypt the data. We can easily track decryptions since we have a different data key for each record.

Installation

Add this line to your application’s Gemfile:

gem "kms_encrypted"

And follow the instructions for your key management service:

AWS KMS
Google Cloud KMS
Vault

AWS KMS

Add this line to your application’s Gemfile:

gem "aws-sdk-kms"

Create an Amazon Web Services account if you don’t have one. KMS works great whether or not you run your infrastructure on AWS.

Create a KMS master key and set it in your environment along with your AWS credentials (dotenv is great for this)

KMS_KEY_ID=arn:aws:kms:...
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

You can also use the alias

KMS_KEY_ID=alias/my-alias

Google Cloud KMS

Add this line to your application’s Gemfile:

gem "google-cloud-kms"

Create a Google Cloud Platform account if you don’t have one. KMS works great whether or not you run your infrastructure on GCP.

Create a KMS key ring and key and set it in your environment along with your GCP credentials (dotenv is great for this)

KMS_KEY_ID=projects/my-project/locations/global/keyRings/my-key-ring/cryptoKeys/my-key

Vault

Add this line to your application’s Gemfile:

gem "vault"

Enable the transit secrets engine

vault secrets enable transit

And create a key

vault write -f transit/keys/my-key derived=true

Set it in your environment along with your Vault credentials (dotenv is great for this)

KMS_KEY_ID=vault/my-key
VAULT_ADDR=http://127.0.0.1:8200
VAULT_TOKEN=secret

Getting Started

Create a migration to add a column for the encrypted KMS data keys

add_column :users, :encrypted_kms_key, :text

And update your model

class User < ApplicationRecord
  has_kms_key

  # Lockbox fields
  has_encrypted :email, key: :kms_key

  # Lockbox files
  encrypts_attached :license, key: :kms_key

  # attr_encrypted fields
  attr_encrypted :email, key: :kms_key
end

For each encrypted attribute, use the kms_key method for its key.

Auditing & Alerting

Context

Encryption context is used in auditing to identify the data being decrypted. This is the model name and id by default. You can customize this with:

class User < ApplicationRecord
  def kms_encryption_context
    {
      model_name: model_name.to_s,
      model_id: id
    }
  end
end

The context is used as part of the encryption and decryption process, so it must be a value that doesn’t change. Otherwise, you won’t be able to decrypt. You can rotate the context without downtime if needed.

Order of Events

Since the default context includes the id, the data key cannot be encrypted until the record has an id. For new records, the default flow is:

Start a database transaction
Insert the record, getting back the id
Call KMS to encrypt the data key, passing the id as part of the context
Update the encrypted_kms_key column
Commit the database transaction

With Postgres, you can avoid a network call inside a transaction with:

class User < ApplicationRecord
  has_kms_key eager_encrypt: :fetch_id
end

This changes the flow to:

Prefetch the id with the Postgres nextval function
Call KMS to encrypt the data key, passing the id as part of the context
Insert the record with the id and encrypted data key

If you don’t need the id from the database for context, you can use:

class User < ApplicationRecord
  has_kms_key eager_encrypt: true
end

AWS KMS

AWS CloudTrail logs all decryption calls. You can view them in the CloudTrail console. Note that it can take 20 minutes for events to show up. You can also use the AWS CLI.

aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=Decrypt

If you haven’t already, enable CloudTrail storage to S3 to ensure events are accessible after 90 days. Later, you can use Amazon Athena and this table structure to query them.

Read more about encryption context here.

Alerting

Set up alerts for suspicious behavior. To get near real-time alerts (20-30 second delay), use CloudWatch Events.

First, create a new SNS topic with a name like "decryptions". We’ll use this shortly.

Next, open CloudWatch Events and create a rule to match “Events by Service”. Choose “Key Management Service (KMS)” as the service name and “AWS API Call via CloudTrail” as the event type. For operations, select “Specific Operations” and enter “Decrypt”.

Select the SNS topic created earlier as the target and save the rule.

To set up an alarm, go to this page in CloudWatch Metrics. Find the rule and check “Invocations”. On the “Graphed Metrics” tab, change the statistic to “Sum” and the period to “1 minute”. Finally, click the bell icon to create an alarm for high number of decryptions.

While the alarm we created isn’t super sophisticated, this setup provides a great foundation for alerting as your organization grows.

You can use the SNS topic or another target to send events to a log provider or SIEM, where can you do more advanced anomaly detection.

You should also use other tools to detect breaches, like an IDS. You can use Amazon GuardDuty if you run infrastructure on AWS.

Google Cloud KMS

Follow the instructions here to set up data access logging. There is not currently a way to see what data is being decrypted, since the additional authenticated data is not logged. For this reason, we recommend another KMS provider.

Vault

Follow the instructions here to set up data access logging.

Note: Vault will only verify this value if derived was set to true when creating the key. If this is not done, the context cannot be trusted.

Context will show up hashed in the audit logs. To get the hash for a record, use:

KmsEncrypted.context_hash(record.kms_encryption_context, path: "file")

The path option should point to your audit device. Common paths are file, syslog, and socket.

Separate Permissions

A great feature of KMS is the ability to grant encryption and decryption permission separately.

Be extremely selective of servers you allow to decrypt.

For servers that can only encrypt, clear out the existing data and data key before assigning new values (otherwise, you’ll get a decryption error).

# Lockbox
user.email_ciphertext = nil
user.encrypted_kms_key = nil

# attr_encrypted
user.encrypted_email = nil
user.encrypted_email_iv = nil
user.encrypted_kms_key = nil

AWS KMS

To encrypt the data, use an IAM policy with:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "EncryptData",
            "Effect": "Allow",
            "Action": "kms:Encrypt",
            "Resource": "arn:aws:kms:..."
        }
    ]
}

To decrypt the data, use an IAM policy with:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DecryptData",
            "Effect": "Allow",
            "Action": "kms:Decrypt",
            "Resource": "arn:aws:kms:..."
        }
    ]
}

Google Cloud KMS

todo: document

Vault

To encrypt the data, use a policy with:

path "transit/encrypt/my-key"
{
  capabilities = ["create", "update"]
}

To decrypt the data, use a policy with:

path "transit/decrypt/my-key"
{
  capabilities = ["create", "update"]
}

Apply a policy with:

vault policy write encrypt encrypt.hcl

And create a token with specific policies with:

vault token create -policy=encrypt -policy=decrypt -no-default-policy

Testing

For testing, you can prevent network calls to KMS by setting:

KMS_KEY_ID=insecure-test-key

In a Rails application, you can also create config/initializers/kms_encrypted.rb with:

KmsEncrypted.key_id = Rails.env.test? ? "insecure-test-key" : ENV["KMS_KEY_ID"]

Key Rotation

Key management services allow you to rotate the master key without any code changes.

For AWS KMS, you can use automatic key rotation
For Google Cloud, use the Google Cloud Console or API
For Vault, use:

vault write -f transit/keys/my-key/rotate

New data will be encrypted with the new master key version. To encrypt existing data with new master key version, run:

User.find_each do |user|
  user.rotate_kms_key!
end

Note: This method does not rotate encrypted files, so avoid calling rotate_kms_key! on models with file uploads for now.

Switching Keys

You can change keys within your current KMS or move to a different KMS without downtime. Update your model:

class User < ApplicationRecord
  has_kms_key version: 2, key_id: ENV["KMS_KEY_ID_V2"],
    previous_versions: {
      1 => {key_id: ENV["KMS_KEY_ID"]}
    }
end

New data will be encrypted with the new key. To update existing data, use:

User.where("encrypted_kms_key NOT LIKE 'v2:%'").find_each do |user|
  user.rotate_kms_key!
end

Once all data is updated, you can remove the previous_versions option.

Switching Context

You can change your encryption context without downtime. Update your model:

class User < ApplicationRecord
  has_kms_key version: 2,
    previous_versions: {
      1 => {key_id: ENV["KMS_KEY_ID"]}
    }

  def kms_encryption_context(version:)
    if version == 1
      # previous context method
    else
      # new context method
    end
  end
end

New data will be encrypted with the new context. To update existing data, use:

User.where("encrypted_kms_key NOT LIKE 'v2:%'").find_each do |user|
  user.rotate_kms_key!
end

Once all data is updated, you can remove the previous_versions option.

Multiple Keys Per Record

You may want to protect different columns with different data keys (or even master keys).

To do this, add another column

add_column :users, :encrypted_kms_key_phone, :text

And update your model

class User < ApplicationRecord
  has_kms_key
  has_kms_key name: :phone, key_id: "..."

  # Lockbox
  has_encrypted :email, key: :kms_key
  has_encrypted :phone, key: :kms_key_phone

  # attr_encrypted
  attr_encrypted :email, key: :kms_key
  attr_encrypted :phone, key: :kms_key_phone
end

To rotate keys, use:

user.rotate_kms_key_phone!

For custom context, use:

class User < ApplicationRecord
  def kms_encryption_context_phone
    # some hash
  end
end

Outside Models

To encrypt and decrypt outside of models, create a box:

kms = KmsEncrypted::Box.new

You can pass key_id, version, and previous_versions if needed.

Encrypt

kms.encrypt(message, context: {model_name: "User", model_id: 123})

Decrypt

kms.decrypt(ciphertext, context: {model_name: "User", model_id: 123})

To securely search encrypted data, check out Blind Index.

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

Report bugs
Fix bugs and submit pull requests
Write, clarify, or fix documentation
Suggest or add new features

To get started with development and testing:

git clone https://github.com/ankane/kms_encrypted.git
cd kms_encrypted
bundle install
bundle exec rake test

KMS Encrypted

How It Works

Installation

AWS KMS

Google Cloud KMS

Vault

Getting Started

Auditing & Alerting

Context

Order of Events

AWS KMS

Alerting

Google Cloud KMS

Vault

Separate Permissions

AWS KMS

Google Cloud KMS

Vault

Testing

Key Rotation

Switching Keys

Switching Context

Multiple Keys Per Record

Outside Models

Related Projects

History

Contributing