Model providers and end clients
Three parties take part in an encrypted inference.
| Party | Holds | Runs |
|---|---|---|
| Model provider (you) | The model, a Belfort API key | belfort-ml |
| End client | The input data, the secret key | belfort-ml-client |
| Belfort | The compiled model, evaluation keys, ciphertexts | The gateway, the compiler, GPU deploys |
The end client has no Belfort account and never receives an API key. It reaches the gateway with a client token you mint, and Belfort's storage on presigned URLs the gateway issues for that token, to download its client code and upload its evaluation keys.
What travels between you and the end client#
One client token. It authenticates the end client to the Belfort gateway for
one model. Belfort does not deliver it: you send it to the end client by
whatever means you already use with them.
model.create_client(label="acme-clinic") opens a key set for that end client on
the model and returns a ClientToken; the label is your name for that end
client. Send its token, which
also names the gateway the end client calls.
The token reaches the gateway's client routes and no others, and only for that one key set. With it, the Belfort ML client:
- Fetches the compile's client code: temporary download links with the digest the gateway verified. It downloads the files, checks them against the digest, and builds them.
- Generates a key pair, names its evaluation keys to the gateway, and uploads them to the temporary upload links it gets back.
- Waits until the gateway has loaded those keys onto the deploy you started for its key set, then sends ciphertexts for inference.
A token opens one key set, which one end client uses; a later run needs the same secret seed or a new token. Another end client needs a new token, and a new deploy: a deploy serves the one key set you start it for.
No model weights reach the end client. Its token downloads only the generated
client code, never the weights or the server code. GET /models/{m_id}/artifacts
returns every file of the compile, the weights included, so do not forward its
links to an end client.
Who sees what#
The secret key, plaintext inputs and decrypted outputs stay on the end client's machine. Belfort holds the compiled model, the evaluation keys and the ciphertexts, none of which reveal the data. See Keys and encryption for the key's lifetime.