Open formula. Browser-based calculations. Source-dated rates. No API keys. No provider login. No tracing SDK. No account required. Input payloads are not stored.
Pricing source date: 2026-05-28 · Build verification: 2026-05-28 · Stale threshold: 30 days. Estimates use published per-token API pricing for the model/provider path shown by the calculator, reviewed on the source date shown. Provider rates change frequently; verify current pricing on the provider's own page before committing to traffic volume, eval cadence, or agent-loop workload shape.
Budgets repeated model-evaluation runs across samples, candidate models, trials, and optional judge passes.
Formula
total_cost = primary_eval_calls * primary_model_cost + judge_eval_calls * judge_model_cost
Primary source register
Eval costs move with model pricing and trial count. Treat recurring CI/eval usage as a monthly cost, not a one-time experiment.
- https://openai.com/pricing
- https://www.anthropic.com/pricing
- https://platform.claude.com/docs/en/about-claude/pricing
- https://platform.claude.com/docs/en/about-claude/models/overview
- https://cloud.google.com/vertex-ai/generative-ai/pricing
- https://www.together.ai/pricing
- https://replicate.com/pricing
- https://fireworks.ai/pricing
- https://groq.com/pricing
- https://deepinfra.com/pricing
- https://aws.amazon.com/bedrock/pricing/
- https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/
Included assumptions
- samples
- models
- trials per sample
- primary input/output tokens
- optional LLM-as-judge pass
Excluded assumptions
- human review cost
- dataset licensing
- annotation tooling
- latency/runtime scheduling cost
Architecture-cost audit
Eval-suite budgetSamples, candidate models, repeated trials, and optional LLM-as-judge pass costs.
Diagnostic output
This tool returns Cost classification, Dominant cost driver, Decision threshold, and Sensitivity. Diagnostic focus: Eval-suite run cost and monthly recurring-eval budget threshold..
Affiliate link. Model Ruler may earn a commission if you sign up. This does not affect what you pay.