Model Routing: When to Use Cheap vs Expensive Models
Model routing sends simple tasks to cheap models and reserves premium models for hard ones—often cutting cost without losing quality.
The cheapest model isn't always the one with the lowest per-token rate. The cheapest is the one that finishes the job with the fewest billed tokens.
The wrong way to compare models
Comparing headline per-token rates is misleading. A cheaper model that needs two refinement passes can cost more than a premium model that finishes in one.
Compare total billed tokens per user action, not the rate card.
A simple routing rule
- Cheap models for extraction, classification, summarization, and routing decisions
- Premium models only for the final draft, complex reasoning, or ambiguous tasks
Most workflows are 80% simple and 20% hard. Route accordingly.
Example
A support workflow: classify → retrieve → draft → polish.
- Classify and retrieve: cheap model
- Draft: cheap model
- Polish the risky 20%: premium model
The result often matches "premium everywhere" at a fraction of the cost.
Guardrails
- Set a confidence threshold to decide routing
- Cap how often the premium model is invoked
- Re-check routing when quality complaints rise
Quick checklist
- Split tasks by complexity
- Route simple steps to cheap models
- Reserve premium models for final quality
Related reading
Set up AI API cost monitoring that catches token waste, retry spikes, and runaway agents before they hit your bill.
Batch APIs offer large discounts for asynchronous jobs. Learn when to route work to batch and when real-time is worth the premium.
Estimate AI API cost from tokens and request volume before shipping, so you don't discover unusable unit economics after launch.
Agents multiply cost through retries, tool chains, and non-converging loops. Learn to measure and cap the runaway parts.