---
description: Configuring and using `Provider`.
---

# `Provider`

## Configuration

```python
import belfort_ml

provider = belfort_ml.Provider()                        # from the environment
provider = belfort_ml.Provider(api_key="blf_live_...")  # or explicit
```

| Argument | Environment | Default |
|---|---|---|
| `api_key` | `BELFORT_API_KEY` | required |
| `base_url` | `BELFORT_BASE_URL` | `https://eml.belfortlabs.com/api/v1` |
| `timeouts` | | `Timeouts()`: 30 s standard, 300 s per upload part, 420 s inference, 10 s poll |
| `max_retries` | | 3 |

Explicit arguments take precedence over the environment. A missing key raises
`ConfigurationError` when the provider is constructed.

Set `BELFORT_BASE_URL` only to point at a different gateway, for example
`http://localhost:8000/api/v1` for a local one. For the gateway serving these
docs:

```bash
export BELFORT_BASE_URL=https://eml.belfortlabs.com/api/v1
```

The provider holds a connection pool. Use it as a context manager or call
`close()`:

```python
with belfort_ml.Provider() as provider:
    ...
```

## Objects

`provider.account.get()` and `provider.credits.get()` return plain dicts.
`provider.models` and `provider.deploys` return objects that carry their own
methods, so ids rarely need to be passed around.

| Object | Returned by | Key attributes |
|---|---|---|
| `Model` | `provider.models.create/get/list` | `id`, `status`, `params`, `active_compile_id`, `level_zero_encryption`, `use_composite_relu` |
| `ClientKeySet` | `model.clients` | `id`, `label`, `status`, `size_bytes` |
| `Deploy` | `model.deploy`, `provider.deploys.get/list` | `id`, `c_id`, `status`, `terminates_at`, `credits_reserved`; `secret_seed` and `client_token` once a developer-mode deploy is ready |
| `LoadedKeys` | `deploy.load_keys` | `k_id` |
| `ClientToken` | `model.create_client` | `token`, `c_id`, `m_id`, `label`, `expires_at` |
| `InferenceResult` | `deploy.infer` | `output`, `request_id`, `latency_ms` |

## Methods

| Method | What it does |
|---|---|
| `belfort_ml.compile(model, example_input, name, output_dir, calibration_inputs=, allow_unsupported_ops=False)` | Local compile to MLIR; returns `Artifacts`. `name` must be a C++ identifier that is not a keyword. Raises `CompileError` on an operation the Belfort compiler does not support; `allow_unsupported_ops=True` writes the MLIR and logs a warning instead. Warns with `MultiCiphertextWarning` when a layer does not fit one ciphertext |
| `provider.account.get()` | Your account, including `mp_id` and credits |
| `provider.credits.get()` | Credit balance |
| `provider.credits.transactions(cursor=, limit=, kind=)` | One page of credit transactions; follow `next_cursor` |
| `provider.models.create(artifacts, name=, wait=True, timeout=3600, level_zero_encryption=False, use_composite_relu=True)` | Register, upload, compile, wait. See [Level-zero encryption](#level-zero-encryption) and [ReLU approximation](#relu-approximation) |
| `provider.models.get(m_id)` / `provider.models.get(name=)` / `provider.models.list()` | Look up models; `list` follows pages lazily |
| `provider.models.get(name=, create_from=artifacts, timeout=3600, level_zero_encryption=False, use_composite_relu=True)` | Return the model of that name, creating it from `artifacts` when missing and waiting until it is ready. A model of that name with a different MLIR, `level_zero_encryption` or `use_composite_relu` raises `ConflictError` `name_taken` |
| `model.refresh()` / `model.wait_until_ready()` | Re-read the compile status |
| `model.delete()` | Delete a model with no in-use deploy, with its uploaded MLIR, its key sets and its stopped deploys |
| `model.create_client(label, ttl_seconds=)` | Open a key set for one end client and mint the token it authenticates to the gateway with, valid 24 hours by default |
| `model.wait_for_keys(key_set, timeout=3600)` | Wait until the end client has uploaded its evaluation keys and the gateway verified them. Raises `KeyMaterialError` on a `failed` or `expired` key set |
| `model.clients(status=)` | The model's key sets, with `id` (the `c_id`), `label`, `status` and `size_bytes` |
| `model.delete_client(c_id)` | Delete a key set that no running deploy serves, which ends its client token |
| `model.deploy(key_set=, max_lifetime=, timeout=, dev_mode=False)` | Start a GPU deploy that serves `key_set` (a `ClientToken`, a `ClientKeySet` or a `c_id`) for its whole life, and wait for it. `dev_mode=True`, without `key_set`, is for testing: see [Developer mode](../concepts/deploys-and-credits.md#developer-mode) |
| `provider.deploys.get(d_id)` / `.list()` / `.stop_all()` | Look up or stop deploys |
| `deploy.load_keys()` / `deploy.unload_keys(loaded)` | Put the deploy's key set on the GPU, or remove it |
| `deploy.infer(loaded, ciphertext)` / `deploy.infer_batch(loaded, [...])` | Run an inference on a ciphertext you hold |
| `deploy.extend(additional)` / `deploy.stop()` / `deploy.refresh()` | Manage the deploy |

`deploy.infer` is for ciphertexts that are already in your hands, for example
test data an end client sent you. Normally the end client calls the gateway
itself with the token from `model.create_client`, so the ciphertext does not
pass through you, and the gateway loads its keys onto the deploy without a
`load_keys` call. `load_keys` is for loading a key set onto a deploy yourself.

Every method that waits (`create`, `get(name=, create_from=)`, `deploy`, `load_keys`) polls
the resource's status, starting every 2 seconds and backing off to 10, and
raises `TimeoutError` when its `timeout` runs out. `wait=False` returns
immediately where it is offered. While a deploy starts, `deploy` writes the
deploy id and each launch step (looking for a GPU instance, booting it,
loading the model, then loading the client's keys if the end client has
uploaded them) to stderr; `progress=False` turns that off.

## Level-zero encryption

`level_zero_encryption=True` compiles the model so that the end client encrypts
each input at the bottom of the modulus chain, and the deploy bootstraps each
input before the computation.

This is a tradeoff: it will make the ciphertext at the client much smaller,
hence reducing upload time and bandwidth usage. The computation will however
take longer as an additional bootstrap operation will be performed at the
beginning. It is not recommended be use it with models that do not require
bootstrapping as it will greatly increase the evaluation keys' size.

## ReLU approximation

Encrypted computation cannot evaluate ReLU exactly, so the compiler replaces
each ReLU with a polynomial over the range that calibration recorded.

With `use_composite_relu=True`, the default, each ReLU is a composite
approximation: three chained polynomials, of degrees 15, 15 and 27. It is accurate, but it uses many
levels per activation, so the model bootstraps more often. More levels and
bootstraps make each inference slower and the evaluation keys larger.

`use_composite_relu=False` uses a single degree-5 polynomial instead. A small
model can then fit without bootstrapping, with much smaller keys. The
approximation is coarser, so compare the encrypted results with the plaintext
model before you use it. The [Quickstart](../getting-started/quickstart.md)
uses it.

These two are the only choices: the degree of the approximation, for ReLU and
for the other approximated operations, cannot be set yet.

## Uploads

`provider.models.create` uploads the MLIR, and the Python file that defines the
model when `artifacts.source_path` is set, to storage through presigned URLs, in
parts, without sending your API key. A failed part is retried; an upload that
still fails raises `UploadError`.

## Logging

The SDK logs to the standard `belfort_ml` logger, which is silent unless you
configure a handler. DEBUG lines contain the method, path, status and request
id, and omit headers and bodies.

```python
import logging
logging.basicConfig(level=logging.INFO)
```