TengoChatDevelopers

Errors and retries

Every error type, what it means, and when to try again.

Every error has an HTTP status and a JSON body: {"error": {"type", "message", "param", "code"}}. The message is written for people; the type is for your code.

Retry or not

429 rate_limitedWait the seconds in Retry-After, then retry.
502 / 503 serverRetry with exponential backoff: 1, 2, 4, 8 seconds.
402 insufficient_creditsDo not retry until credits are added.
429 spend_limitThe key's monthly limit: raise it, or wait for the new month.
400 invalid_requestFix the request; param names the field.
401 authenticationCheck the key is sent and not revoked.
404 not_foundThe model, agent, thread or file is not on this account.

A retry helper

JavaScript
async function call(url, init, tries = 4) {
  for (let i = 0; ; i++) {
    const res = await fetch(url, init);
    if (res.ok) return res.json();
    const retry = res.status === 429 && (await res.clone().json()).error?.type === "rate_limited";
    if ((retry || res.status >= 500) && i < tries) {
      const wait = Number(res.headers.get("Retry-After")) || 2 ** i;
      await new Promise((r) => setTimeout(r, wait * 1000));
      continue;
    }
    throw new Error((await res.json()).error?.message ?? res.statusText);
  }
}