Running an AI feature costs money across four distinct areas: token charges on every API call, hosting for the server routes that proxy those calls, storage and logs for the data you need to audit or improve the model, and any monitoring tools on top. The number that surprises most SMB owners is not the per-call cost - it is the aggregate once volume picks up. A product sending 2,000 requests per day, each carrying 800 input tokens and 300 output tokens, is moving roughly 2.2 million tokens through the API daily. At Anthropic's list prices on 19 October 2026 that is about 1.6 million input and 0.6 million output tokens a day, which works out to roughly $0.46 on Claude Haiku 5.5, $9.20 on Claude Sonnet 5.5, $18.80 on Claude Opus 5.5 and $46 on Claude Fable 5.1 - the same traffic, a hundredfold range, depending entirely on which model you chose, before you have added anything else. Understanding where the bill comes from is the first step to keeping it manageable.
The Four Line Items on a Typical AI Bill
Token costs are the headline number. Every provider charges for input tokens (the prompt and any context you send) and output tokens (the model's response). Output tokens are priced higher than input tokens - on every current Claude model listed on Anthropic's pricing page, output costs five times as much as input - so both a bloated prompt and a long answer show up on the bill. Output length is often harder to control: the max_tokens parameter sets the absolute maximum the model will generate, and a generous value leaves room for far more than the user needs.
Hosting and compute come second. If you use a hosted API, your compute cost is the API bill itself. But you still need a server-side proxy - a Next.js API route, a Vercel function, or an Express endpoint that holds your key and routes requests. That proxy runs on your hosting platform, and under load it costs money too.
Logs and storage are easy to miss. Every AI request and response should be logged so you can audit outputs, debug failures, and run evals later. A month of logs for a busy feature can be gigabytes. Structured logs on a managed service like Supabase or a cloud log store add a line to the bill.
Third-party tooling rounds it out. Moderation APIs, embeddings for retrieval-augmented generation, vector databases, and eval platforms each add their own usage-based charge. These are optional but commonly stacked on top of the base model cost.
Where the Bill Leaks Silently
The most common source of silent overspend is prompt verbosity. Sending a 2,000-token system prompt on every call when 400 tokens would do the same job multiplies your input cost by five. Teams copy a working prompt from a demo, expand it during development, and never revisit it once the feature ships.
Lack of output length control is the second leak. If the output limit is set high, a model can keep generating until it decides to stop. Users rarely need 1,500 tokens when 200 would answer their question. Setting a sensible upper limit on outputs cuts cost and often improves quality at the same time.
Retry logic without exponential back-off is the third. When a request fails, a naive retry loop hammers the API with duplicate calls, multiplying your cost and triggering rate limits. If you have connected an AI chatbot to your website, check how retries are handled in your integration layer.
Ignoring cheaper model tiers for routine tasks is the fourth. A frontier model trained for complex reasoning is expensive per token. Many tasks - summarising short text, classifying a support ticket, extracting a field from a form - do not need that capability. Running them on a smaller model in the same provider's family can cut those specific costs substantially; measure the saving on your own traffic before budgeting around a specific percentage.
Three Levers That Reduce Cost Without Hurting Quality
Prompt caching
Several model providers allow you to cache a static portion of your prompt so it is not re-tokenised and re-charged on every call. If your system prompt is 1,500 tokens and never changes between users, caching it means you pay for those tokens once rather than on every request. As of October 2026, both Anthropic and OpenAI document prompt caching. On Anthropic's price list a cache hit costs 5% of the standard input price (for example $0.10 instead of $2 per million tokens on Claude Sonnet 5.5), while writing to the cache costs more than a normal input token, so it pays off for prompts that repeat. Check each provider's current documentation for eligibility and pricing, since rates and terms change frequently.
Batching
Batch APIs allow you to queue non-urgent requests and receive results within a longer time window instead of in real time. Both Anthropic and OpenAI currently price batch requests at a 50% discount, with OpenAI quoting a 24-hour turnaround. It is useful for overnight report generation, bulk classification, and any workload where a few minutes of latency is acceptable. It is not the right fit for user-facing features that need an immediate response.
Model routing
Model routing means using a lightweight model for requests that do not need heavy reasoning, and reserving a more capable model for complex tasks. A two-tier setup - a fast, cheap model handles most traffic, a frontier model handles the rest - can substantially reduce average cost per call. The hard part is classifying which requests belong to which tier without adding latency. Start by looking at your logs: the majority of real user requests are often simpler than the edge cases you focused on during development.
Setting a Budget Before You Launch
Before you go live, estimate your daily token volume by running 100 representative test requests and measuring the average input and output token count. Multiply by your expected daily request volume to get a monthly token budget. Then check the current pricing page for your provider - prices have changed frequently since 2023, so always use the live figure rather than one you read months ago. Add 30 to 40 percent headroom for growth and for the prompt inflation that tends to happen as you refine features after launch.
Set a hard spend limit or budget alert in your provider's dashboard before launch, if it offers one. A cap does not protect you from every scenario - some providers pause the API rather than your entire product - but it gives you early warning rather than a surprise bill at month end. For more on managing API spend and abuse after launch, see our post on rate limits and abuse prevention.
After Launch: What to Monitor
Cost per user is the most useful signal. Divide your monthly AI spend by your monthly active users. If cost per user is rising faster than revenue per user, the feature is not yet profitable at scale. Tracking this monthly is far more useful than watching the aggregate invoice, because aggregate spend naturally rises with user growth.
Token count per request is the second signal. If average input or output token count trends upward over time, a prompt change or a product change has made the feature more expensive to run. Catch it early and it is easy to fix. Catch it at month three and you may be re-architecting under revenue pressure. Building an eval loop that tests AI output quality alongside cost is the right long-term answer - the AI evals post covers how small teams approach this without a dedicated ML team.
AI feature running costs are predictable and largely controllable once you understand the levers. The biggest mistakes happen when teams ship first and budget later. If you are planning an AI feature and want help scoping the cost model and the architecture before you commit to a provider and a model, talk to us about an AI feature review. We look at the technical design, the token budget, and the fall-through paths - so the bill on day 90 is not a surprise.
Common questions
How much does it cost to run an AI feature?
An AI feature costs money across four areas: token charges, hosting for the server routes that proxy calls, storage and logs, and monitoring tools. The same 2,000 daily requests ranged from roughly $0.46 to $46 a day depending on model, at Anthropic's list prices on 19 October 2026.
How can I reduce my AI API bill without hurting quality?
Three levers help: prompt caching for static prompt portions, batching non-urgent requests at a 50% discount, and routing simple requests to a cheaper model. Also trim verbose prompts and set sensible output limits. Check each provider's current documentation, since rates and terms change frequently.
When should I get help scoping the cost of an AI feature?
Before committing to a provider and model, estimate daily token volume from 100 representative test requests and add 30 to 40 percent headroom. If you want help scoping the cost model and architecture, Zovintra offers an AI feature review covering the technical design, token budget and fall-through paths.
Sources
- Pricing - Anthropic Claude Platform Docs, accessed 19 Oct 2026
- Prompt caching - Anthropic Claude Platform Docs, accessed 19 Oct 2026
- Prompt caching - OpenAI API docs, accessed 19 Oct 2026
- Batch API - OpenAI API docs, accessed 19 Oct 2026
- Create a Message (max_tokens) - Anthropic Claude Platform Docs, accessed 19 Oct 2026
