LLM API rate limits (HTTP 429): how do I implement exponential backoff with jitter and honor the Retry-After header for Claude and OpenAI, and which 429s should not be retried?

Cause: A 429 means a request, token or concurrency limit was hit. Claude's limits are per organization, use a token bucket (capacity replenishes continuously rather than resetting on a fixed clock), and are measured in requests per minute plus input and output tokens per minute; a sharp traffic increase can also trigger acceleration limits. OpenAI's guide says limits are per organization and list common triggers such as loops, concurrent calls and keys shared with other apps. Fix (best first): 1. Honor Retry-After. Both providers document it: if the header is present, wait at least that many seconds before retrying; earlier retries fail. If it is missing, use exponential backoff with jitter (for example delay = random(0, min(cap, base * 2^attempt))) and cap the number of attempts. 2. Let the official SDK retry first. The Claude SDKs retry transient failures (connection errors, rate limits, 5xx) twice by default with backoff and respect retry-after; configure max_retries (maxRetries in TypeScript). If you add your own retry layer on top, account for the SDK's retries so you do not multiply them, and respect long server delays. 3. Do not retry non-transient 429s. A Claude monthly spend-cap 429 has no retry-after header and keeps failing until access resumes (error details code enforced_spend_limit_reached); OpenAI's billing and quota errors such as credit_balance_exhausted or spend-limit errors also will not clear by retrying. Raise limits or add credit instead. Claude returns 400 (not 429) when a spend limit you set yourself is reached. 4. Smooth traffic: use a client-side limiter or queue, cap concurrency, ramp up gradually, and read the rate-limit response headers (Claude's anthropic-ratelimit-* headers) to slow down before hitting zero. 5. Reduce usage: cache prompts (for most Claude models cached input tokens do not count toward ITPM), batch non-urgent work with the Batch API, and use smaller models where possible. Request higher limits through the console if needed. 6. Treat 500/503/529 overloaded errors as retryable with the same backoff. Verify: Under load, retries wait the advertised time, success rate recovers without a retry storm, and quota or spend-cap errors surface as alerts instead of looping. Sources: https://platform.claude.com/docs/en/api/rate-limits https://developers.openai.com/api/docs/guides/error-codes https://platform.claude.com/docs/en/api/errors