Rate limits - Yelu AI API

Yelu applies request limits to protect platform reliability. Limits can vary by account group, model, deployment configuration, and whether a request succeeds. The dashboard and your account configuration are authoritative for your effective limits.

When a request is limited

The API returns HTTP 429 Too Many Requests. Parse the OpenAI-compatible error body, wait, and retry only when the operation is safe to repeat.

{
  "error": {
    "message": "Rate limit reached. Please retry later.",
    "type": "rate_limit_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

Retry strategy

Use exponential backoff with random jitter and a strict retry cap. A practical sequence is approximately 0.5, 1, 2, and 4 seconds, randomized so concurrent workers do not retry together.

for delay in 1 2 4 8; do
  status=$(curl -sS -o response.json -w "%{http_code}" \
    https://api.yelu.ai/v1/chat/completions \
    -H "Authorization: Bearer $YELU_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}')
  [ "$status" -lt 429 ] && break
  [ "$status" -gt 429 ] && break
  sleep "$delay"
done
cat response.json

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function requestWithBackoff(body, attempts = 4) {
  for (let attempt = 0; attempt < attempts; attempt += 1) {
    const response = await fetch('https://api.yelu.ai/v1/chat/completions', {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${process.env.YELU_API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(body),
    });
    if (response.ok) return response.json();
    if (response.status !== 429 || attempt === attempts - 1) {
      throw new Error(await response.text());
    }
    const delay = 500 * 2 ** attempt + Math.random() * 250;
    await sleep(delay);
  }
}

import os
import random
import time
import requests

def request_with_backoff(payload, attempts=4):
    for attempt in range(attempts):
        response = requests.post(
            "https://api.yelu.ai/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.environ['YELU_API_KEY']}"},
            json=payload,
            timeout=60,
        )
        if response.ok:
            return response.json()
        if response.status_code != 429 or attempt == attempts - 1:
            response.raise_for_status()
        time.sleep(0.5 * (2 ** attempt) + random.uniform(0, 0.25))

Reduce limit pressure

Queue bursts and control worker concurrency instead of releasing all work at once.
Cache deterministic application results where your data policy allows it.
Stream long generations to improve user-perceived latency, but remember that streaming does not necessarily reduce quota usage.
Use separate keys for independent workloads so one batch job does not starve interactive traffic.
Bound retry attempts; repeated failures consume capacity and can amplify incidents.
Avoid retrying 400, 401, 403, and 404 responses without changing the request.

Retryable status codes

Status	Retry?	Guidance
`408`	Usually	Retry with backoff if the client can safely repeat the operation.
`409`	Sometimes	Retry only if the error indicates a temporary conflict.
`429`	Yes	Back off with jitter and reduce concurrency.
`500`, `502`, `503`, `504`	Yes	Use a small retry budget; switch to a fallback model only if your product permits it.

See Errors for the complete handling model.

​When a request is limited

​Retry strategy

​Reduce limit pressure

​Retryable status codes

When a request is limited

Retry strategy

Reduce limit pressure

Retryable status codes