The right LLM for a product feature is rarely the most capable model you can name. It is the model that satisfies your quality threshold at a cost you can sustain, with a latency your users will tolerate, and without coupling your codebase so tightly that a price change or model retirement causes a crisis.
Start With the Actual Requirement, Not a Benchmark
LLM benchmarks measure performance on standardised tasks such as coding, reasoning and translation. Your feature almost certainly is not one of those tasks. A customer support classifier that needs to distinguish between five categories of issue does not need the same model as a code generation feature. Using benchmark rankings as a purchasing decision is like choosing a database by its TPC-C score.
The practical starting point is a sample of real inputs and a rubric for acceptable outputs. Anthropic's own model selection guide calls a good evaluation set the most important step. As a rule of thumb, gather a few dozen to a hundred representative examples of the task - ideally from actual user behaviour if you have it, or from manually constructed examples if you are pre-launch. Define what good and bad output looks like in concrete terms. This becomes your evaluation dataset, and it is more useful than any public benchmark.
The evals approach described for small teams applies directly here: before you pick a model, you need a way to measure whether the model is doing the job. Without an eval, switching models later becomes a manual guessing exercise.
The Four Dimensions That Matter
Quality
Quality means the model produces outputs that satisfy your rubric, not that it scores well on MMLU. For classification tasks, quality is measurable: run your evaluation set and check accuracy. For generative tasks - summarisation, drafting, code generation - quality is more subjective and usually requires human review of a sample.
In our experience, current frontier models are often close enough in quality for many text tasks that quality alone does not decide the choice, but test this on your own data. Where quality can diverge: long context (how well the model reasons over a 100k-token document), code generation, structured output reliability (does it follow a JSON schema consistently?), and multilingual accuracy.
Cost
LLM pricing is per input and output token. As an illustration using the list prices on Anthropic's models overview for Claude Sonnet 5.5 ($2 per million input tokens, $10 per million output tokens), a feature that sends a 2,000-token prompt and receives a 500-token response costs about $0.009 per call. At 10,000 calls per day that is about $90 per day or roughly $2,700 per month - from one feature. Check current prices before you budget; they change. The running costs of AI features grow non-linearly with usage because prompt length often grows with context window size.
Cost optimisation strategies: use a smaller model for the first pass and escalate to a larger model only for cases that fail a confidence threshold; cache responses for identical or near-identical inputs; compress system prompts (shorter prompts cost less); use batch APIs for non-real-time workloads: OpenAI's Batch API offers a 50% discount with a 24-hour window, and Google's Gemini Batch API is priced at half the standard rate with a 24-hour target turnaround.
Model providers change prices and retire models. Do not build a business case around today's prices without a contingency for price changes in either direction.
Latency
For user-facing features, latency is the time from API call to complete response. Streaming (receiving tokens as they are generated) helps perceived latency even when total time is unchanged. For a chat interface, streaming is almost always worth implementing. For a background feature that returns a result in an async job, total latency matters less.
Larger models tend to be slower than smaller ones. Anthropic's comparison table rates Claude Haiku 5.5 as fastest and Sonnet 5.5 as fast, with Opus 5.5 moderate and Fable 5.1 slower. Time-to-first-token (TTFT) is more important than total time for streaming use cases. If your feature needs a response in a few hundred milliseconds without streaming, measure first: a large model may not meet that requirement, and you may need a smaller model or a cached response.
Lock-In
Lock-in is the least discussed dimension and often the most costly. It has three components: API format lock-in, model capability lock-in, and data lock-in.
API format lock-in is addressable. Writing an abstraction layer that accepts model names as configuration rather than hardcoding them is a small upfront investment that pays off at every model transition.
Model capability lock-in is harder. If your prompts are tuned to a specific model's behaviour - its tendency to follow certain formats, its treatment of edge cases, its refusal patterns - switching requires re-evaluation and often re-tuning. This is not a reason to avoid frontier models, but it is a reason to run evals before and after every model switch.
Data lock-in applies when you are using fine-tuning or retrieval systems tied to a specific provider. If your retrieval setup uses proprietary embeddings from one provider, migrating embeddings to a different provider means re-embedding all your documents. This cost is often invisible at the start and significant at scale.
A Decision Matrix
- Low latency, simple classification or extraction: a smaller, cheaper tier from your provider (Anthropic suggests starting with Haiku for efficiency-first projects) is probably enough. Test first.
- High-quality generation with large context: a frontier model. Budget and monitor costs from day one.
- Background batch processing: use the batch API if available; OpenAI and Google both list a 50% discount, and Anthropic lists 50% off for its Batch API.
- Regulated or sensitive data: check data residency and DPA terms for every provider. This is not optional and it affects which providers are available to you.
- If you cannot switch providers without a significant re-engineering effort: you have a lock-in problem to address before you scale.
The Question Nobody Asks Until It Is Too Late
What happens when this model is retired? Every major provider retires models. Anthropic, for example, states it gives at least 60 days' notice for publicly released models; check each provider's own policy. A team that has not abstracted the model name in its configuration will spend several days of engineering time at the worst moment, usually when traffic is growing.
The model retirement planning checklist covers the operational side of this. From a selection perspective, the question is: can you test a replacement model against your eval set and deploy it in less than a week? If the answer is no, the coupling is too tight.
Getting to a Decision
Pick two or three candidate models based on the matrix above. Run your evaluation set against each. Measure quality (your rubric), cost (token counts multiplied by current pricing), and latency (average and 95th percentile). The model that passes your quality threshold at the lowest cost is the starting choice. Reassess when usage grows past your initial estimates.
If you are building an AI feature into a product and want a structured review of the architecture - including model selection, cost controls, and fallback behaviour - that kind of advisory engagement is within our scope. Book a scoped AI feature review for an independent view on your approach.
Common questions
How do I choose the right LLM for my product feature?
Start with a few dozen to a hundred real examples and a rubric for good output, then run two or three candidate models against it. The model that passes your quality threshold at the lowest cost and acceptable latency is the starting choice.
How much does an LLM API call cost?
Pricing is per input and output token. As an illustration, at $2 per million input tokens and $10 per million output tokens, a 2,000-token prompt with a 500-token response costs about $0.009 per call. Check current prices before you budget; they change.
When should I get an independent review of my AI feature architecture?
Consider one when you are building an AI feature and want a structured view of model selection, cost controls and fallback behaviour. Zovintra offers a scoped AI feature review that gives an independent opinion on your approach.
Sources
- Models overview - Anthropic documentation, accessed 19 Oct 2026
- Choosing the right model - Anthropic documentation, accessed 19 Oct 2026
- Model deprecations - Anthropic documentation, accessed 19 Oct 2026
- Batch API - OpenAI documentation, accessed 19 Oct 2026
- Batch API - Google AI for Developers, accessed 19 Oct 2026
