What is a keystone fernet token?

Keystone fernet token format is based on the cryptographic authentication method – Fernet. Fernet is an implementation of Symmetric Key Encryption. Symmetric key encryption is a cryptographic mechanism that uses the same cryptographic key to encrypt plaintext and the same cryptographic key to decrypt ciphertext. Fernet authentication method also supports multiple keys where it takes a list of symmetric keys, performs all encryption using the first key in a list and attempts to decrypt using all the keys from that list. Keystone fernet token looks like:


gAAAAABWHXT73mGHg90PE6rmS-6aeYYvdErvO1RCWbDBrM5JV6L-eGEkz9cv8598DWWF5LZH5buzYM6PmUk3w9PHd4j6zs9L0_nvqZAGOrA4gLjhE10MLk00_Qy-IIPMQ6kxjsphYVLP1uBUNyh-s4hq76-KGNUqAcYgLyN8DtgoifDseSZKNl8

How do I configure keystone to use fernet token?

Fernet token can be configured by setting the token provider along with certain fields under [fernet_token] section in keystone.conf :

[token]
      	provider = keystone.token.providers.fernet.Provider
[fernet_tokens]
        # key repository where the fernet keys are stored
	key_repository = /etc/keystone/fernet-keys/
        # maximum number of keys in key repository
	max_active_keys =  # default is 3

Fernet keys are stored in key repository which is set to /etc/keystone/fernet-keys/. Fernet key file is named based on integers starting from 0.

$ ls /etc/keystone/fernet-keys
0 1 2 3 4

There are three types of key files:

  • Type 1: Primary Key – Primary key is used for both encrypting and decrypting fernet tokens. Primary key file is the one named with the highest index.
  • Type 2: Secondary Key – Secondary key is used only for decrypting fernet tokens. Secondary key file is named lower than the highest index but greater than the lowest index (lowest index is always 0).
  • Type 3: Staged Key – Staged key is similar to secondary key in which it is used only for decrypting fernet tokens. And staged key differs from the secondary key in which it is the next in line to become primary key whereas secondary key was a primary key prior to applying one or more rotation cycles. Staged key file is the one named with lowest index (of 0).

In summary, fernet tokens are encrypted using Primary Key and decrypted using a list of fernet keys from the key repository.

What is the fernet key format?

The fernet key is a base64 encoding of Signing Key (16 bytes) and Encrypting Key (16 bytes). An example of fernet key is MmcGs0_iRH-GybC41AcxdtgvgIi4kk3T94bAqoL7l-k= and this is how you can generate it:

>>> import base64
>>> import os
>>>
>>> b_key = os.urandom(32) # the fernet key in binary
>>> b_key
'2g\x06\xb3O\xe2D\x7f\x86\xc9\xb0\xb8\xd4\x071v\xd8/\x80\x88\xb8\x92M\xd3\xf7\x86\xc0\xaa\x82\xfb\x97\xe9'
>>>
>>> b_key[:16] # signing key is the first 16 bytes of the fernet key
'2g\x06\xb3O\xe2D\x7f\x86\xc9\xb0\xb8\xd4\x071v'
>>>
>>> b_key[16:] # encrypting key is the last 16 bytes of the fernet key
'\xd8/\x80\x88\xb8\x92M\xd3\xf7\x86\xc0\xaa\x82\xfb\x97\xe9'
>>>
>>> key = base64.urlsafe_b64encode(b_key) # base64 encoded fernet key
>>> key
'MmcGs0_iRH-GybC41AcxdtgvgIi4kk3T94bAqoL7l-k='

How are the fernet keys rotated?

Lance Bragstad and Matt Fischer have written blog post on the Fernet Key Rotation.

Here is an attempt to demonstrate Fernet Key Rotation using some animation: Fernet Key Rotation

Let’s start with fernet-setup using keystone-manage utility which creates two keys in the key repository. Key file 1 which is the primary key and key file 0 which is the staged key, without any secondary key.

Consider it’s now time to rotate, identify the highest index, which is 2 in this case. During the key rotation, staged key 0 becomes primary key with a file name 2. Previous primary key (1) becomes secondary key with the same file name 1. And a new staged key (file named 0) is introduced.

During next key rotation cycle, the highest index is 3. Staged key 0 becomes primary key with a file name 3. Previous primary key (2) becomes secondary key with the same file name 2. Previous secondary key remains secondary key with the same file name 1. And a new staged key (file named with 0) is introduced. If we had set max_active_keys to 3, secondary key 1 is deleted during this round of key rotation.

Do I have to rotate Fernet keys at the same time across multiple keystone deployments?

No, Fernet tokens are created using primary key and the tokens are decrypted and validated using the list of keys from fernet key repository.

Let’s take an example of two keystone deployments (us-west and us-east) with both repositories set to:

$ ls /etc/keystone/fernet-keys
0 1 2

In this deployment, 2 is the primary key and fernet tokens are created using this primary key. Fernet tokens are decrypted using all three keys in order 2, 1, and 0.

Now, when we rotate fernet keys in us-west and have the repository set to:

$ ls /etc/keystone/fernet-keys
0 1 2 3

With this configuration in us-west, 3 becomes primary key and fernet tokens are created using this primary key. When keystone in us-west receives fernet token from us-east (which is generated using key file 2), us-west can still validate that token as it decrypts token with all four keys in order 3, 2, 1, and 0. Now, when keystone in us-east receives fernet token from us-west (which is generated using key file 3), us-east can still validate that token as the primary key (3) in us-west is the staged key (0) in us-east, and keystone decrypts token with all three keys in order 2, 1, and 0.

Keystone supports unsynchronized fernet key rotation but it is not recommended to delay fernet key rotation in practice, it can be delayed until the configuration management tool is finished rotating keys in all data centers but should not be delayed by more than one rotation cycle.

How is keystone fernet token generated?

Keystone fernet token is the base64 encoding of a collection of the following fields:

  • Fernet Format Version (0x80) – the only version available – 8 bits
  • Current Timestamp – 64 bits
    >>> import time
    >>> time.time()
    1444771059.637071
    >>> current_time = int(time.time())
    >>> current_time
    1444771067
    >>> import struct
    >>> struct.pack(">Q", current_time)
    '\x00\x00\x00\x00V\x1dt\xfb

    > denotes big-endian
    Q denotes unsigned long long

  • Initialization Vector (IV) – 128 bits
    >>> iv = os.urandom(16)
    >>> iv
    '\xdea\x87\x83\xdd\x0f\x13\xaa\xe6K\xee\x9ay\x86/t'
  • Ciphertext: The original keystone payload depending on the scope of the token is padded to meet the block size and encrypted to create a ciphertext. Lets look at how keystone payload looks like for project scoped token. Keystone payload for project scoped token is a collection of (Version, User_ID, Methods, Project_ID, Expiration Time, Audit IDs)
    • Version: Fixed versioning by keystone:
      • Unscoped Payload : 0
      • Domain Scoped Payload : 1
      • Project Scoped Payload : 2
      • Trust Scoped Payload : 3
      • Federated:
        • Unscoped Payload : 4
        • Domain Scoped Payload : 6
        • Project Scoped Payload : 5
    • User ID: Byte representation of User ID, uuid.UUID(user_id).bytes
    • Methods: Integer representation of list of methods, a unique integer is assigned to each method, for example, 1 to oauth1, 2 to password, 3 to token, etc. Now the sum of list of methods in the token generation request is calculated, for example, for “methods”: [“password”], result is 2. For “methods”: [“password”, “token”], result is 2 + 3 = 5.
    • Project ID: Byte representation of Project ID, uuid.UUID(project_id).bytes
    • Expiration Time: Timestamp integer of expiration time in UTC
    • Audit IDs: Byte representation of URL-Safe string, restoring padding (==) at the end of the string
    >>> from cryptography.hazmat.primitives import hashes, padding
    >>> from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
    >>> from cryptography.hazmat.backends import default_backend
    >>>
    >>> padder = padding.PKCS7(algorithms.AES.block_size).padder()
    >>> padder.__dict__ # padder is set to 128 block size
    {'block_size': 128, '_buffer': ''}
    >>>
    >>> project_scoped_payload = '\x96\x02\xb0\x134\xf3\xed~\xb2H;\x91\xb8\x19+\xa0C\xb5\x80\x02\xb0B=E\xcd\xde\xc8Ap\xbe6^\x0b1\xa1\xb1_\xcbA\xd5\x87P\x02\xb4C\xd9\x91\xb0}oA&\xd3fCu\x95z\\\xbd\xd8{\x89\xbc' # A sample project scoped payload
    >>>
    >>> padded_data = padder.update(project_scoped_payload) + padder.finalize() # apply padding to meet block size
    >>> padded_data
    '\x96\x02\xb0\x134\xf3\xed~\xb2H;\x91\xb8\x19+\xa0C\xb5\x80\x02\xb0B=E\xcd\xde\xc8Ap\xbe6^\x0b1\xa1\xb1_\xcbA\xd5\x87P\x02\xb4C\xd9\x91\xb0}oA&\xd3fCu\x95z\\\xbd\xd8{\x89\xbc\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10'
    >>>
    >>> backend = default_backend()
    >>> e_key = b_key[16:] # encrypting key is the last 16 bytes of fernet key
    >>> encryptor = Cipher(algorithms.AES(e_key), modes.CBC(iv), backend).encryptor() # AES encryption is used in CBC mode 
    >>>
    >>> ciphertext = encryptor.update(padded_data) + encryptor.finalize() # encrypt padded keystone payload
    >>> ciphertext
    'J\xef;TBY\xb0\xc1\xac\xceIW\xa2\xfexa$\xcf\xd7/\xf3\x9f|\re\x85\xe4\xb6G\xe5\xbb\xb3`\xce\x8f\x99I7\xc3\xd3\xc7w\x88\xfa\xce\xcfK\xd3\xf9\xef\xa9\x90\x06:\xb08\x80\xb8\xe1\x13]\x0c.M4\xfd\x0c\xbe \x83\xccC\xa91\x8e\xcaaaR\xcf'
  • HMAC: 256-bit SHA256 HMAC with signing key, concatenating all of the above fields: (Version, Timestamp, IV, Ciphertext)
    >>> s_key = b_key[:16] # signing key is the first 16 bytes of fernet key
    >>>
    >>> from cryptography.hazmat.primitives.hmac import HMAC
    >>> h = HMAC(s_key, hashes.SHA256(), backend) # SHA256 hash is used while creating HMAC
    >>>
    >>> basic_parts = (b"\x80" + struct.pack(">Q", current_time) + iv + ciphertext) # HMAC is a combination of Fernet Version, Timestamp, IV, and Ciphertext
    >>> basic_parts
    '\x80\x00\x00\x00\x00V\x1dt\xfb\xdea\x87\x83\xdd\x0f\x13\xaa\xe6K\xee\x9ay\x86/tJ\xef;TBY\xb0\xc1\xac\xceIW\xa2\xfexa$\xcf\xd7/\xf3\x9f|\re\x85\xe4\xb6G\xe5\xbb\xb3`\xce\x8f\x99I7\xc3\xd3\xc7w\x88\xfa\xce\xcfK\xd3\xf9\xef\xa9\x90\x06:\xb08\x80\xb8\xe1\x13]\x0c.M4\xfd\x0c\xbe \x83\xccC\xa91\x8e\xcaaaR\xcf'
    >>>
    >>> h.update(basic_parts)
    >>> hmac = h.finalize() # Calculate HMAC of combination of Fernet Version, Timestamp, IV, and Ciphertext 
    >>> hmac
    '\xd6\xe0T7(~\xb3\x88j\xef\xaf\x8a\x18\xd5*\x01\xc6 /#|\x0e\xd8(\x89\xf0\xecy&J6_'

    Finally, Fernet Token is Base 64 URL Safe encoded, combination of Version, Timestamp, IV, Ciphertext, and HMAC.

    >>> fernet = base64.urlsafe_b64encode(basic_parts + hmac) # Fernet token is base64 URL-Safe encoding of Version, Timestamp, IV, Ciphertext, and HMAC
    >>> fernet
    'gAAAAABWHXT73mGHg90PE6rmS-6aeYYvdErvO1RCWbDBrM5JV6L-eGEkz9cv8598DWWF5LZH5buzYM6PmUk3w9PHd4j6zs9L0_nvqZAGOrA4gLjhE10MLk00_Qy-IIPMQ6kxjsphYVLP1uBUNyh-s4hq76-KGNUqAcYgLyN8DtgoifDseSZKNl8='
    >>>
    >>> fernet.rstrip('=') # strip off the last character to make it compliant with HTTP header
    'gAAAAABWHXT73mGHg90PE6rmS-6aeYYvdErvO1RCWbDBrM5JV6L-eGEkz9cv8598DWWF5LZH5buzYM6PmUk3w9PHd4j6zs9L0_nvqZAGOrA4gLjhE10MLk00_Qy-IIPMQ6kxjsphYVLP1uBUNyh-s4hq76-KGNUqAcYgLyN8DtgoifDseSZKNl8'
    

1 Comment on "Deep Dive into Keystone Fernet Tokens"

  1. Miguel Zuniga January 12, 2016

    No wonder why everything sound so familiar 😛

Join The Discussion

Your email address will not be published. Required fields are marked *