OpenAI API 429: A Retry, Queue, and Recovery Runbook

Problem: a 429 can turn one request into duplicate work

A rate-limit response is not proof that the request failed safely. A caller may retry after the server accepted work, or multiple workers may retry the same task together. The result can be duplicate messages, double writes, and an overloaded queue.

This guide is for teams operating an OpenAI-backed API, background worker, or automation workflow. It is not a recipe for bypassing limits. If the account has no remaining quota, retries cannot fix it.

Recommendation: retry only queued, idempotent work

Put non-interactive work behind a durable queue. Give each job an idempotency key, a maximum attempt count, and a deadline. For a user-facing request, return a clear retry-later state instead of holding the connection indefinitely.

Do not add a general retry wrapper around actions that send email, issue refunds, or write external records unless the downstream action is idempotent. A manual review or a compensating action is safer than a blind replay.

System breakdown: five controls, not one loop

1. Classifier: record status, response body, request ID when available, model, and estimated input size.
2. Admission control: cap concurrent work before calls reach the API.
3. Retry policy: use exponential backoff with jitter and a fixed attempt or time budget.
4. Idempotency store: mark a business operation as pending, completed, or needs review.
5. Recovery queue: move exhausted jobs to a visible queue with the failure evidence attached.

The OpenAI error-code guide distinguishes rate-limit and quota problems from request validation errors. The rate-limit guide explains that limits apply to requests and tokens, so reducing concurrency alone may not solve a token-heavy workload. Sources checked: August 30, 2026.

Design trade-offs: fast response versus controlled completion

Start with a queue when work can be delayed. Keep inline retries for a bounded, read-only request where the caller can safely receive a retry-later response.

Step-by-step: implement a bounded worker

1. Define the job contract

Store job_id, idempotency_key, input version, attempt count, and deadline before calling the model. Never use the prompt text itself as an idempotency key; the same task can legitimately have revised inputs.

2. Limit concurrency

Begin with a deliberately low worker count and raise it only after observing the request and token limits for the selected model. Keep interactive traffic separate from batch traffic so a batch spike does not consume every available slot.

3. Retry only transient cases

async function runWithBudget(run, job) { for (let attempt = 0; attempt < 3; attempt += 1) { try { return await run(); } catch (error) { const retryable = error?.status === 429 || error?.status >= 500; if (!retryable || Date.now() > job.deadline) throw error; const waitMs = Math.min(8000, 500 * 2 ** attempt) * (0.5 + Math.random()); await new Promise(resolve => setTimeout(resolve, waitMs)); } } throw new Error('retry budget exhausted'); }

This is a starting pattern, not a universal setting. Set the attempt count and deadline from the user promise, the queue visibility timeout, and the cost of a duplicate result. Respect any server-provided retry timing where your client exposes it.

4. Commit the business action once

Write the model result and the idempotency key in the same transaction when possible. If the model call succeeds but the commit fails, mark the job as ambiguous and investigate before replaying an external action.

5. Route exhausted jobs

After the budget expires, persist the error evidence and move the job to a recovery queue. Do not silently discard it and do not create an unbounded retry schedule.

Common failures and recovery

• Every worker retries at once: add jitter and reduce admission before raising retry attempts.
• A quota problem is treated as temporary: alert the account owner and pause the queue; retrying only adds traffic.
• A retry sends the same message twice: require an idempotency record before the downstream action.
• One large prompt consumes the token budget: measure input size, split the job, or move it to a slower queue.
• Workers cannot tell whether completion happened: retain the provider request reference and use an explicit ambiguous state.

Costs and operations

Retries can still consume capacity. Track attempts per job, time spent waiting, input and output tokens when available, queue depth, and the share of jobs sent to recovery. A falling error rate is not sufficient if queue age keeps increasing.

Do not promise a fixed cost reduction from this design. Its value is control: the team can see which work was delayed, retried, completed once, or requires a decision.

Limits and alternatives

This runbook does not increase an account limit, guarantee a response time, or make an irreversible workflow safe by itself. For low-volume prototypes, a simple synchronous path with a clear user retry message may be better than operating a queue. For large offline workloads, the official batch options may be a better fit than pushing real-time traffic through more workers.

FAQ

Should I retry every 429?

No. First distinguish a temporary rate limit from a quota or configuration problem. Retry only work that is safe to replay and still within its deadline.

How many retries are correct?

There is no universal number. Use a small budget tied to the user-facing deadline or queue policy, then inspect the recovery rate and queue age.

Can a queue remove the need for rate-limit monitoring?

No. A queue absorbs bursts; it does not create capacity. Monitor limits, job age, and recovery outcomes.

Implementation checklist

• Classify 429, quota, validation, and server errors separately.
• Assign an idempotency key before external side effects.
• Set a concurrency cap for each workload class.
• Use jittered, bounded retries with a deadline.
• Persist exhausted and ambiguous jobs for review.
• Measure queue age, attempts, and duplicate-prevention outcomes.
• Test a quota pause and a worker restart before production rollout.

Definition of done: a test job can be rate-limited, delayed, retried within budget, completed at most once, and recovered with evidence when it cannot complete.

Further study

Read the official error-code guide and rate-limit guide, then compare this operating model with batch and prompt-caching cost controls, human approval queues, and context-engineering workflow controls.