Errors and retries

Errors#

Every exception the SDK raises descends from belfort_ml.BelfortMLError.

flowchart LR
    B[BelfortMLError] --> Cfg["ConfigurationError<br/>no API key, or a base URL that is not http(s)"]
    B --> Cmp["CompileError<br/>local or Belfort-side compile failure"]
    B --> Up["UploadError<br/>an upload part failed after retries"]
    B --> Dn["DownloadError<br/>reading compiled artifacts failed"]
    B --> Km["KeyMaterialError<br/>a key directory holds a file that is not an evaluation key"]
    B --> To["TimeoutError<br/>a wait ran past its timeout"]
    B --> Cn["ConnectionError<br/>the gateway was unreachable after retries"]
    B --> Api["APIError<br/>the gateway returned an error"]
    Api --> V["ValidationError<br/>400, 413"]
    Api --> Au["AuthenticationError<br/>401"]
    Api --> Ic["InsufficientCredits<br/>402, with .required and .available"]
    Api --> Pe["PermissionError<br/>403"]
    Api --> Nf["NotFoundError<br/>404"]
    Api --> Co["ConflictError<br/>409"]
    Co --> Dl["DeployLimitReached<br/>409 deploy_limit_reached, with .limit and .running"]
    Api --> Rl["RateLimitError<br/>429, with .retry_after"]
    Api --> Su["ServiceUnavailable<br/>503"]

APIError carries message, code, detail, status_code and request_id. Include the request_id when you report a problem to Belfort.

Python

try:
    deploy = model.deploy(key_set=minted, max_lifetime=timedelta(hours=2))
except belfort_ml.InsufficientCredits as e:
    print(f"need {e.required} credits, have {e.available}")
except belfort_ml.APIError as e:
    print(e.code, e.message, e.request_id)

Common ConflictError codes:

Code	Cause
`name_taken`	Creating a model with a name your account already uses, or `models.get(name=, create_from=)` with a different MLIR than that model's
`deploy_running`	Deleting a model while one of its deploys is starting or ready
`key_set_not_ready`	Loading a key set that has not been verified
`key_set_unusable`	Starting a deploy for a key set whose keys failed verification or expired
`key_loaded`	Deleting a key set that a running deploy serves
`key_set_mismatch`	Loading another key set than the one the deploy serves
`keys_already_loaded`	Loading the deploy's key set while it holds or loads it
`deploy_not_ready`	Using a deploy that is still starting or already stopping
`deploy_limit_reached`	Starting a deploy while the account's one deploy is starting or ready; raised as `DeployLimitReached`

Retries#

The SDK retries connection errors, timeouts, and 429, 500, 502 and 504 responses up to max_retries times (3 by default), with exponential backoff and jitter capped at 30 seconds. A 429's Retry-After takes precedence over the computed backoff. Other 4xx responses are not retried. A read or write timeout can come after the gateway acted on the request, so after one only GET requests are sent again; a POST /deploys that timed out raises ConnectionError instead of starting a second deploy.

Inference is never retried automatically, since a retry would run on the GPU again. To retry one yourself, pass the same request_id to deploy.infer within five minutes: the gateway returns the stored result instead of running it again. Choose the request_id yourself on the first call, since a call that fails does not return the one deploy.infer generated.

model.create_client() is not retried either: each call opens a key set, so a retry after a lost answer would leave a key set whose token nobody holds. If it fails, call it again; an unused key set expires with its token.

Capacity#

A deploy's runtime handles one inference at a time; concurrent requests wait for it. A request waits up to 30 seconds for the runtime, then fails with 429 and retry_after: 30. Each inference gets 300 seconds of GPU time.