The real cost of running AI agents in production
Chatbots are cheap. Agents are not.
A chatbot sends a user message, gets a response, displays it. Maybe 2,000 tokens per exchange. An agent reads files, calls tools, retries on errors, re-sends the entire conversation every step, and does this 20–60 times per task. Same API, completely different economics.
If you’re budgeting for AI agents the same way you budget for a chatbot, you’re underestimating by 10–50x.
Token consumption: chatbot vs. agent
Section titled “Token consumption: chatbot vs. agent”We measured token consumption across three workload types, each running for one hour:
The coding agent consumed 52x more tokens than a simple chatbot in the same time period. And this is normal — the agent was doing useful work the entire time.
Why agents cost so much
Section titled “Why agents cost so much”Three architectural properties of agents make them expensive:
1. Context accumulation
Section titled “1. Context accumulation”Every agent step appends tool outputs to the conversation. The LLM re-processes the entire conversation on each step. If the agent reads a 3,000-token file at step 5, that file gets re-sent at steps 6, 7, 8… all the way to the end.
For a 40-step task, one file read costs: 3,000 tokens × 35 remaining steps = 105,000 tokens in re-transmission.
This is why agent token consumption grows quadratically, not linearly.
2. System prompt overhead
Section titled “2. System prompt overhead”Agent frameworks use large system prompts — OpenClaw’s is ~9,600 tokens, CrewAI’s varies by agent configuration. This prompt is sent with every request. Over 40 steps, the system prompt alone costs 384,000 tokens.
3. Error retry loops
Section titled “3. Error retry loops”When a tool call fails, the agent retries. Each retry sends the full context plus the error message. Three retries on a 30K-token context wastes 90K tokens with no productive output.
Without a retry cap, this can run indefinitely — always bound agents with a retry cap and a maximum iteration count.
Monthly cost by model and framework
Section titled “Monthly cost by model and framework”Assuming one developer running 15 agent tasks per day, 22 working days per month, ~500K tokens per task (80% input / 20% output, at each vendor’s list price — live prices here):
| Model | Cost/task | Daily (×15) | Monthly |
|---|---|---|---|
| Claude Fable 5 | $9.00 | $135.00 | $2,970 |
| GPT-5.5 | $5.00 | $75.00 | $1,650 |
| Claude Opus 4.8 | $4.50 | $67.50 | $1,485 |
| GLM 5.2 (open) | $1.00 | $15.00 | $330 |
| DeepSeek V4 Flash (open) | $0.08 | $1.26 | $28 |
| CheapestInference (full day) | — | — | from $48.45/mo flat |
A team of 5 developers each running 15 tasks/day on Claude Fable 5 spends $14,850/month. The same team on flat-rate via CheapestInference pays a fixed monthly subscription per seat (from $48.45/mo for a reserved daily time block) — no matter how many tokens those agents burn. That’s an order-of-magnitude reduction.
Four strategies to cut agent inference costs
Section titled “Four strategies to cut agent inference costs”1. Switch to open-source models
Section titled “1. Switch to open-source models”Open-weight models like Kimi K2.6 and MiniMax M3 now sit at parity with Gemini 3.1 Pro on SWE-bench Verified, and GLM 5.2 outscores GPT-5.5 on SWE-bench Pro — at a half to a sixth of the per-token price. Full data: the Which-LLM guide and the State of Open Weights report.
2. Route by task complexity
Section titled “2. Route by task complexity”Not every agent step needs a frontier model. File reads, simple classifications, and formatting don’t need 685B parameters. Use a small model for easy steps and a large model for hard ones. Full guide: Building a multi-model architecture.
3. Give each agent its own key
Section titled “3. Give each agent its own key”Give each agent its own API key so one runaway agent can’t starve the others. On a time-block subscription each key gets unlimited usage during its reserved hours, so you isolate workloads without juggling per-token allocations.
4. Use flat-rate pricing
Section titled “4. Use flat-rate pricing”Per-token pricing penalizes the exact patterns agents use: large contexts, many steps, retries. Flat-rate pricing makes all of that free. During your reserved time blocks your agent can use the full context window and retry freely without increasing the bill — reserve all three blocks for 24/7 coverage.
The math that matters
Section titled “The math that matters”Here’s the equation most teams miss:
Agent cost = tokens_per_step × steps × cost_per_tokenMost optimization focuses on cost_per_token — switching to a cheaper model. But tokens_per_step grows with context (quadratic), and steps is unpredictable. Optimizing only one variable leaves the other two working against you.
Flat-rate pricing eliminates all three variables from your bill. The cost is the subscription. Period.
We serve six open-weight models across two flat-rate pools — Kimi K2.7, Kimi K2.6, GLM 5.2, and MiniMax M3 in the Frontier pool (from $48.45/mo), DeepSeek V4 Flash and MiMo v2.5 in the Core pool (from $12.74/mo) — with unlimited time-block subscriptions: no token counting, no budget caps during your reserved hours. Reserve 1–3 daily 8-hour blocks and your agent’s token consumption never becomes your problem. Get started or see plans.