LLM latency budget calculator
How long will users wait? Set an output length, pick a model to highlight, and compare estimated response times across major providers.
Output length
Tokens in the model's reply (not your prompt).
Highlight model
Estimated response time
Sorted fastest to slowest. TTFT = time to first token; Gen = remaining generation time.
| Model | Provider | TTFT | Gen | Total | Relative |
|---|---|---|---|---|---|
Streaming vs waiting
Everything runs in your browser. Numbers are rough mid-2026 estimates — not benchmarks from your exact prompt or region.
About this tool
Latency is the UX cost nobody puts in a spreadsheet. This calculator combines time-to-first-token (TTFT) with output throughput to estimate total wait time: total ≈ TTFT + outputTokens ÷ tokens/sec.
Figures are ballpark mid-2026 estimates from public benchmarks and vendor docs. They vary with prompt size, queue depth, caching, region, and whether the model is warm. Use them to compare orders of magnitude — not to promise SLAs.
Groq and similar speed-focused hosts trade model size and reasoning depth for raw tokens/sec. Frontier models on OpenAI or Anthropic are slower per token but often need fewer tokens to finish the job — latency and quality are not the same tradeoff.
Read more
- An introduction to Cloudflare Workers — edge inference and low-latency deploys
- A hands-on guide to The Agency, a collection of AI agents
- Will AI replace web developers?
- On AI doomerism
Free books, courses, and software
Get my programming books, course editions, and complete software.
Get the download library →