How do you keep LLM costs under control in production?

Question

Our AI feature's bill tripled last month. What are the first things to look at?

Sara Okafor · Accepted Answer

In order of impact:
- **Log tokens per request** by feature, so you know where the money goes.
- Use a smaller model for easy steps (classification, routing) and the big one only where quality needs it.
- Use prompt caching for long, repeated system prompts.
- Trim the context: send the 5 most relevant chunks, not 50.
- Set per-user rate limits.

How do you keep LLM costs under control in production?

1 Answer

Log in to view all 1 Answer