Inference Cost Calculator
Estimate LLM inference expenses from requests, latency, and token volumes online for free. Model prompt caching discounts, batch API savings, retry overhead, and concurrent throughput capacity — then compare costs and latency across 14 models. Runs 100% in your browser.
Inference Cost Calculator
Estimate LLM inference expenses from request volume, token sizes, latency SLA, and throughput. Model prompt caching discounts, batch API savings, retry overhead, and concurrent capacity — then compare costs across 14 major models. Runs 100% in your browser.
Real-time user-facing chat with sub-2s latency SLA
Token Sizes per Request
Request Volume & Latency
Cost Optimizations
Projected Monthly Inference Cost
Cost / Request
$0.000330
Cost / 1K Req
$0.3300
Est. Latency
3.8s
Max RPS
13.3
Per-Request Cost Breakdown
Model Cost & Latency Comparison
Sorted by lowest monthly cost| Model | $/req | Monthly Cost | Latency | SLA |
|---|---|---|---|---|
Llama 3.1 8BBudget Groq/Together · 600 tok/s · TTFT 80ms | $0.00006800 | $41.62 | 663ms | ✓ Met |
Mistral Small 3.2Budget Mistral · 160 tok/s · TTFT 160ms | $0.000185 | $113.22 | 2.3s | ✗ Breach |
GPT-4o miniBudget Selected OpenAI · 100 tok/s · TTFT 250ms | $0.000330 | $201.96 | 3.8s | ✗ Breach |
DeepSeek-V3Budget DeepSeek · 90 tok/s · TTFT 300ms | $0.000601 | $367.81 | 4.2s | ✗ Breach |
Llama 3.3 70BBudget Groq/Togther · 250 tok/s · TTFT 120ms | $0.000749 | $458.08 | 1.5s | ✗ Breach |
Gemini 2.5 FlashBudget Google · 150 tok/s · TTFT 180ms | $0.001115 | $682.38 | 2.5s | ✗ Breach |
Grok 4.3 xAI · 80 tok/s · TTFT 350ms | $0.001875 | $1,147.50 | 4.7s | ✗ Breach |
Claude 3.5 HaikuBudget Anthropic · 130 tok/s · TTFT 200ms | $0.002040 | $1,248.48 | 2.9s | ✗ Breach |
o4-mini OpenAI · 50 tok/s · TTFT 600ms | $0.002420 | $1,481.04 | 7.6s | ✗ Breach |
Mistral Large 3 Mistral · 70 tok/s · TTFT 380ms | $0.003700 | $2,264.40 | 5.4s | ✗ Breach |
o3Capable OpenAI · 45 tok/s · TTFT 700ms | $0.004400 | $2,692.80 | 8.5s | ✗ Breach |
Gemini 2.5 Pro Google · 65 tok/s · TTFT 450ms | $0.004500 | $2,754.00 | 5.8s | ✗ Breach |
GPT-4oCapable OpenAI · 60 tok/s · TTFT 400ms | $0.005500 | $3,366.00 | 6.2s | ✗ Breach |
Claude Sonnet 4.6Capable Anthropic · 55 tok/s · TTFT 500ms | $0.007650 | $4,681.80 | 6.9s | ✗ Breach |
Why Use Our Inference Cost Calculator?
Real-Time Inference Cost Estimates
Change any input — model, token size, request volume, cache hit rate — and watch your cost per request, period forecast, estimated latency, and max throughput update instantly. The inference cost calculator processes everything locally as you type.
Fully Private & Browser-Based
Your workload parameters, request volumes, and token sizes never leave your device. The inference cost calculator runs entirely client-side with no server uploads, no sign-up required, and 100% free — safe for confidential production planning.
Latency & Throughput Modelling
Beyond cost, the inference cost calculator estimates end-to-end response latency from time-to-first-token and output generation speed benchmarks. It checks your SLA target, computes maximum sustainable RPS from your concurrency settings, and flags capacity gaps.
14-Model Cost & Latency Comparison
Compare inference expenses and latency estimates across 14 models from OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, and Llama simultaneously. Each row shows cost per request, period forecast, estimated response time, and SLA pass/fail status.
Common Use Cases for Inference Cost Calculator
Production Inference Budget Planning
Before launching a feature, use the inference cost calculator to forecast monthly LLM spend. Enter your expected daily request volume and token sizes to project accurate API budgets and avoid surprise bills at scale.
Model Selection for Cost vs. Latency
Compare inference cost and estimated response time across 14 models simultaneously. Find the cheapest model that still meets your latency SLA — whether that is a sub-200ms chatbot or a 30-second batch summarizer.
SLA & Throughput Capacity Planning
Validate that your chosen model can sustain your required requests per second given your concurrency budget. The inference cost calculator flags SLA breaches and capacity gaps before you hit them in production.
Prompt Caching ROI Calculation
Quantify how much prompt caching saves on your specific workload. Set your cache hit percentage and see the exact cost delta per request and over your billing period — useful for justifying the cache implementation effort.
Batch vs. Real-Time Trade-off Analysis
Model the financial impact of routing latency-insensitive workloads through the Batch API. Set the percentage of requests eligible for batch processing and see how the 50% discount reduces your total inference spend.
Retry & Error Rate Cost Modelling
Account for real-world reliability when planning inference budgets. Enter your expected retry rate to include the cost of failed requests that must be retried — a commonly overlooked factor in production cost estimates.
Understanding LLM Inference Costs
What is an Inference Cost Calculator?
An inference cost calculator is a tool that estimates the expense of running live LLM requests — called inference — at a given scale. Unlike training costs (a one-time GPU expense) or embedding costs (bulk indexing), inference is the per-request cost you pay every time a user or pipeline sends a prompt and receives a response. Inference is charged based on input tokens (the prompt) and output tokens (the generated response), measured per million tokens. Our inference cost calculator online goes beyond simple token math — it also models latency SLA compliance, maximum throughput capacity, prompt caching discounts, batch API savings, and retry overhead, so you get an accurate picture of your true inference spend before going to production.
How Our Inference Cost Calculator Works
- Select a Workload Preset: Choose from six realistic presets — Live Chatbot, RAG Query, Batch Summary, Bulk Classifier, Code Assistant, or Custom. Each preset pre-fills token sizes, request volume, concurrency, and a latency SLA target appropriate for that use case.
- Configure Token Sizes & Volume: Enter your input and output tokens per request using the number fields or sliders, set your daily request volume and concurrent request capacity, and specify your P95 latency SLA target in milliseconds.
- Apply Cost Optimizations: Set your prompt cache hit percentage (cached input tokens cost ~25% of the standard rate), the percentage of requests eligible for batch API processing (typically 50% discount), and your expected retry rate to model real-world error overhead.
- Review Cost, Latency & Comparison: The calculator shows cost per request, period forecast, estimated end-to-end latency, max sustainable RPS, and a full 14-model comparison table sorted by cost — with SLA pass/fail status for each model at your workload parameters.
Key Inference Cost Metrics Explained
- Time-to-First-Token (TTFT): The latency between sending a request and receiving the first token of the response. TTFT determines the perceived responsiveness of your application and varies significantly by model and provider — from ~80ms for small quantized models to 700ms+ for frontier reasoning models.
- Output Generation Speed (tokens/sec): How fast the model streams response tokens after the first token arrives. Longer responses with slow generation speed add significant tail latency. The estimated end-to-end latency is TTFT + (output tokens ÷ tokens-per-second).
- Prompt Caching: Many providers cache the KV state of repeated prompt prefixes (like system instructions or document context) so subsequent requests with the same prefix are served at 25% of the standard input token rate. This is highly effective for applications with a large, static system prompt or document context.
- Batch API Discount: Providers including OpenAI, Anthropic, and Google offer asynchronous batch processing at approximately 50% off standard rates for non-latency-sensitive workloads like bulk classification, offline summarization, or background data enrichment.
Privacy, Security & Availability
The inference cost calculator processes all inputs and calculations entirely within your web browser using JavaScript. Your workload parameters, token volumes, request rates, and cost projections are never transmitted to any server, stored in any database, or shared with any third party. The tool is 100% free with no account creation or sign-up required. There are no usage limits — run as many scenarios as you need without any restrictions.
Related Tools
OpenAI Cost Calculator
Estimate API costs across OpenAI models using input/output tokens. Include model selector, token estimator, monthly usage projections, and pricing breakdown.
Claude Cost Calculator
Calculate Anthropic Claude API usage costs online. Estimate prompt and completion expenses with support for Claude 3.5, Opus, Sonnet, Haiku, prompt caching, Batch API, and model cost comparisons.
Gemini Cost Calculator
Estimate Google Gemini API costs based on input and output tokens. Support model comparisons (Gemini 1.5, 2.5, 3.x) and calculate monthly, daily, and annual API pricing forecasts.
DeepSeek Cost Calculator
Estimate DeepSeek API costs based on input, output, and cached prompt tokens. Compare DeepSeek-V4-Flash and DeepSeek-V4-Pro pricing structures online.
Frequently Asked Questions About Inference Cost Calculator
An inference cost calculator estimates how much you pay to run live LLM requests at a given scale. It takes your input token count, output token count, daily request volume, and model selection to project cost per request and a daily, monthly, or yearly total. Our tool also models latency SLA compliance, throughput capacity, prompt caching, batch API discounts, and retry overhead — all locally in your browser with no sign-up required.
Training cost is a one-time expense to build or fine-tune a model using GPU compute. Inference cost is the recurring per-request expense you pay every time a user or pipeline sends a prompt and receives a response. For most production applications, inference is the dominant ongoing AI spend — it scales directly with your user base and request volume.
Yes. The inference cost calculator runs entirely client-side in your browser using JavaScript. Your token sizes, request volumes, latency targets, and cost projections are processed locally on your device and never transmitted to any server or stored anywhere. Your data stays 100% private throughout your session.
End-to-end latency is estimated as: Time-to-First-Token (TTFT) + (output tokens ÷ output generation speed in tokens/sec). For example, a model with 300ms TTFT generating 500 output tokens at 100 tokens/sec has an estimated latency of 300ms + 5,000ms = 5.3 seconds. TTFT and generation speed benchmarks in the calculator are representative medians — actual values vary by provider region, server load, and request size.
TTFT is the time between sending a request and receiving the very first token of the response. It determines how quickly your application can start displaying output to users. Low TTFT (under 200ms) is critical for interactive chat applications. For batch processing, TTFT matters less. Smaller and more quantized models typically have lower TTFT than large frontier models.
Prompt caching stores the key-value (KV) attention state of a repeated prompt prefix — like a long system instruction, a reference document, or a codebase — so subsequent requests using the same prefix do not reprocess those tokens from scratch. Cached input tokens are charged at approximately 25% of the standard input rate. This is highly effective when your system prompt or context is large and shared across many requests.
Use the Batch API for workloads that are not latency-sensitive: bulk document summarization, offline data enrichment, large-scale classification, nightly report generation, or background embedding jobs. The Batch API routes requests through an asynchronous queue processed within 24 hours (varies by provider), and charges approximately 50% of the standard real-time rate. Real-time inference is required for any user-facing interactive feature.
A 2–5% retry rate is typical for production LLM API workloads, accounting for rate limit errors, transient timeouts, and provider-side 5xx errors. High-throughput applications calling frequently at or near rate limits may see higher retry rates (5–15%). The retry rate adds proportionally to your total request count and cost — a 5% retry rate increases effective requests by 5%.