---
description: GPU deploys, lifetimes and how credits are spent.
---

# Deploys and credits

## Deploys

A deploy is one GPU instance serving one compiled model. Each deploy starts a
fresh instance, which takes a few minutes.

```python
from datetime import timedelta

minted = model.create_client(label="acme-clinic")
# hand minted.token to the client

with model.deploy(key_set=minted, max_lifetime=timedelta(hours=1)) as deploy:
    ...  # keep the deploy up while the client runs
```

| Argument | Default | Range |
|---|---|---|
| `key_set` | required, unless `dev_mode=True` | The key set the deploy serves: a `ClientToken`, a registered `ClientKeySet`, or its `c_id` |
| `dev_mode` | `False` | `True` starts a [developer-mode](#developer-mode) deploy; pass no `key_set` with it |
| `max_lifetime` | required | 30 minutes to 5 hours |
| `idle_timeout` | 900 s | 300 s to 3600 s; the gateway refuses a value outside that range |
| `instance_type` | `rtx-pro-6000` | `rtx-pro-6000` only (1x NVIDIA RTX PRO 6000 Blackwell) |
| `timeout` | 600 s | How long the SDK waits for the instance to become ready |

A deploy stops at the first of:

- leaving the `with` block, or `deploy.stop()`
- `max_lifetime` after it started
- `provider.deploys.stop_all()`

The `with` form stops the instance even when your code raises. If the instance
does not start within `timeout`, `model.deploy()` stops it and raises
`belfort_ml.TimeoutError`. A Ctrl-C while the instance starts stops it too; if
that stop fails, it prints the deploy id and a `python -c` command that stops
it, since the instance keeps billing.
A Python process that is killed does not stop its deploy; it runs until
`max_lifetime` unless you call `provider.deploys.stop_all()`.

A deploy serves one key set for its whole life: the one you name in `key_set`.
A key set holds the evaluation keys of one private key, so the deploy holds one
set of weights and one set of evaluation keys, and never takes another key set.
The gateway loads the key set's keys once the deploy is ready and the end
client has uploaded them. If the keys are in when the deploy turns ready,
`model.deploy` starts that load itself and returns once the GPU holds them. A
load that fails there only logs a warning, since the end client's next poll
tries it again. The keys stay there until the Belfort ML client ends, you
unload them, or the deploy stops.

To keep a deploy running longer, `deploy.extend(timedelta(minutes=30))`. The
total lifetime is capped at 5 hours from start.

`deploy.terminates_at` says when a deploy will stop. `provider.deploys.list()` and
the console show what is running.

An account runs one deploy at a time. Since a deploy serves one key set, that
is one end client at a time. While a deploy is starting or ready,
`model.deploy()` raises `belfort_ml.DeployLimitReached` (a `ConflictError`, with
`.limit` and `.running`); stop that deploy first. A deploy that is stopping
does not count.

## Developer mode

Developer mode skips the generation and upload of the end client's evaluation
keys, which can take minutes for a large model. Use it to try a model, not to serve one.

```python
with model.deploy(dev_mode=True, max_lifetime=timedelta(minutes=30)) as deploy:
    deploy.client_token  # for the end client, as BELFORT_CLIENT_TOKEN
    deploy.secret_seed   # 64 hex characters, as BELFORT_CLIENT_SECRET_SEED
```

The gateway opens a key set and chooses its secret seed, and the GPU generates
the evaluation keys from it. Once the deploy is ready, it reports
`secret_seed` and a `client_token` for that key set. An end client started with
both resumes the key set, as in
[The secret key's lifetime](keys-and-encryption.md#the-secret-keys-lifetime),
and starts inferring without uploading keys.

> **Belfort can decrypt**
>
> The gateway stores the secret seed, so Belfort can decrypt everything a
> developer-mode deploy serves. Never send real data to one.

To reuse the same seed on a later deploy, pass that deploy's `c_id` as
`key_set`, without `dev_mode`; the deploy runs in developer mode because its
key set does. The key set and its tokens expire 24 hours after the first
deploy, and the gateway deletes the seed when the key set expires or the end
client releases it. Passing both `key_set` and `dev_mode=True`, or neither, raises
`ValueError`.

## Running continuously

A deploy lives at most 5 hours. Serving an end client around the clock means
starting a new deploy for its key set when the last one ends:

```python title="serve.py"
import time
from datetime import timedelta

while True:
    with model.deploy(key_set=minted, max_lifetime=timedelta(hours=5)) as deploy:
        while deploy.refresh().status == "ready":
            time.sleep(60)
```

What happens to the end client across deploys:

- Its key set outlives the deploy. When the end client next polls its status,
  the gateway loads the key set onto the next ready deploy that serves it.
- An `infer()` that finds no deploy holding its keys waits for the next deploy
  and sends the same ciphertext again, if the first attempt cannot have run.
  Otherwise it reports the outcome as unknown.
- The token must outlive the deploys: mint it with a `ttl_seconds` long
  enough. When it expires, the gateway deletes the key set's keys.
- A long-running `Client` needs no new token until its token expires, after
  7 days at most. `belfort-ml-client infer` needs a new token for every run
  unless every run sets the same `BELFORT_CLIENT_SECRET_SEED`.

The end client cannot run inferences while the next deploy starts, which takes a
few minutes, and while its keys load. An account runs one deploy at a time, so
the next deploy starts only once the current one is stopping. A 5-hour deploy
reserves 30 credits, three times the grant of a new account, so continuous
serving needs more credits than the grant.

## Credits

Credits pay for deploy time. Compiling, uploading keys and running inferences
are free.

| | |
|---|---|
| 1 credit | 10 minutes of deploy time, rounded up |
| New account | 10 credits (100 minutes) |
| More credits | **Request more credits** on the console's Credits page |

When a deploy starts, `ceil(max_lifetime / 10 min)` credits are reserved, so the
shortest deploy, 30 minutes, reserves 3; if your balance is too low,
`model.deploy` raises `InsufficientCredits` (with `.required` and `.available`)
and nothing starts. When it stops, you are charged for the time it ran from
becoming ready, rounded up to 10 minutes, and the rest is released; a deploy
that fails to start costs nothing. `extend` reserves the difference. Example:
`max_lifetime=timedelta(hours=1)` reserves 6 credits; stopped after 25 minutes,
it costs 3. `provider.credits.get()["credit_minutes"]` gives the length of a
credit.

```python
provider.credits.get()           # {"granted": 10, "used": 3, "reserved": 6, "available": 1, "credit_minutes": 10}
provider.credits.transactions()  # one page, newest first
```