Problem
Every call uses the same model, the same effort and huge prompts. The bill grows faster than usage.
Solution
Prompt and answer caching at the entrance; a router picks model and effort level per task type; cost is measured per completed task.
When to use
- →High call volume
- →A mix of simple and complex tasks
- →AI budget under pressure
## Trade-offs
- +Cost drops without losing quality
- +Lower latency on simple tasks
- −Wrong routing degrades quality
- −Requires evaluation per route