Keys and encryption

The keys#

The Belfort ML client generates an FHE key pair when it starts the client binary for a model.

The set of evaluation keys is large, from hundreds of megabytes to several gigabytes depending on the model, and is stored as one file per key: <n>.evk files and one <n>.s2d file. The compiler decides which keys a model needs.

Key sets#

A key set is one end client's evaluation keys, registered against one model and one compile. model.create_client opens one and mints the token the end client uses to reach it:

minted = model.create_client(label="acme-clinic")

The key set starts in awaiting_keys. On that token, the Belfort ML client names the evaluation keys it generated, and the gateway checks the names and returns one presigned upload URL per key. The URLs need no Belfort credential, so the client uploads to them directly; they expire after an hour. Keys are named once per token. Until the upload is verified, naming the same keys with the same sizes again returns fresh links for them, so a lost answer does not end the run. The Belfort ML client does not ask for new links, so with it a key set that takes longer than an hour to upload fails, and the end client needs a new token and a fresh key pair. The client then has the gateway verify that every named key arrived, which moves the set to ready, and the gateway loads it onto the deploy you started for it with model.deploy(key_set=minted, ...).

A key set is the evaluation keys of one private key. A deploy serves the one key set you name when you start it, and no other. Today the secret key stays in the Belfort ML client process that generated it, so one end client uses a deploy.

label is your reference for the end client. It is the only information about the end client that Belfort stores.

Key sets belong to the model's compile#

Keys are generated under the encryption parameters of the model's compile, and the Belfort ML client fetches that compile's code.

The secret key's lifetime#

The secret key lives in the memory of the client binary the Belfort ML client starts. It derives from a 32-byte secret seed (client.secret_seed, or BELFORT_CLIENT_SECRET_SEED). A later process on the same token that passes the seed back restores the same secret key and skips the key upload; without it, the next run needs a new token. Whoever holds the seed can decrypt the end client's data.

The one exception is a developer-mode deploy, for testing: the gateway chooses the seed and the GPU generates the evaluation keys, so Belfort can decrypt.

Belfort ML clients never share a secret key. Each one runs on its own token, with its own key pair and key set, and the gateway treats it as a separate end client. A deploy holds one key set at a time, so clients of the same model use a deploy one after the other, or one deploy each when they run at the same time.

Ciphertexts#

One inference sends one ciphertext holding the whole input, and returns one ciphertext holding the whole output. Its size depends on the encryption parameters the compiler chose.

CKKS is approximate: decrypted results differ from the plaintext model's by a small error. Approximated activations add their own error on top, which grows quickly for inputs outside the calibrated range.

Client tokens#

The end client authenticates to the gateway with a token you mint:

minted = model.create_client(label="acme-clinic", ttl_seconds=86400)
minted.token        # send to the end client; it names the gateway too
minted.expires_at

A client token reaches the gateway's client routes for one key set: the client code, the key upload, the key set's status, inference on the deploy that holds the keys, and ending the client. It reaches none of your account's other routes. It expires after ttl_seconds: 86400 (24 hours) by default, 60 to 604800 (7 days). It has to cover the end client's whole run: the download, the build, the key upload, the wait for a deploy and the inferences. It ends earlier when the end client releases its key set, when you delete the key set with model.delete_client(c_id) (stop the deploys that serve it first), or when you delete the model.

When to mint a new token#

A token belongs to one key set, not to a deploy. Every deploy you start with key_set=minted (or its c_id) serves that key set, so one token works across any number of deploys while it is valid and the end client still holds its secret key: a Client that keeps running, or a later process that passes the same secret seed. Mint a new token:

• for each new end client,
• when the token expires,
• when the end client lost its secret seed, or ran belfort-ml-client infer without BELFORT_CLIENT_SECRET_SEED, which releases the key set at exit.