The first time I checked the Anthropic usage dashboard for Claient after a good week of signups, I did the thing everyone does: I stared at the number, refreshed, and stared again. Nothing was broken. The bot was working exactly as designed. That was the problem. Every design decision that made the bot good — remembering the conversation, running a qualification pass, checking tool results — was also a design decision that made it expensive, and I'd shipped all of them without once asking what they cost per message.
This isn't a "the API is too pricy" post. Anthropic's per-token pricing is fine. It's a post about the four ways I was quietly paying for the same tokens more than once, and the one bug that could have paid for them infinitely. In order of how much money each one actually returned.
Why the bill grows faster than the conversation does
LLMs are stateless. Every call sends the entire conversation back in, not just the new message, because the model has no memory between requests. So turn ten of a WhatsApp thread doesn't cost "one message's worth" of input tokens, it costs one message plus the other nine you already paid to send the previous nine times:
| Turn | Tokens sent as input (rough) | Cumulative billed input |
|---|---|---|
| 1 | 400 | 400 |
| 5 | 1,600 | ~5,600 |
| 15 | 4,800 | ~40,000 |
Nobody designs for this on purpose. You write messages.push(newTurn), ship it, and the shape looks fine in every test because your test conversations are three messages long. Production conversations are not three messages long. A lead who's genuinely interested will happily go fifteen or twenty turns deep, and on the naive implementation you're billed for the whole history, every single time, for a system prompt and tool definitions that haven't changed since message one.
Lever 1: prompt caching, for the part that never changes
The system prompt, the tool schema, the few-shot examples, the business's tone-of-voice block: none of that changes between turn one and turn twenty. Anthropic's prompt caching lets you mark a prefix of the request as cacheable, so instead of paying full price to re-process it every call, you pay a small write cost once and a fraction of the input rate on every read for the next five minutes:
const message = await anthropic.messages.create({
model: "claude-haiku-4-5",
max_tokens: 400,
system: [
{
type: "text",
text: SYSTEM_PROMPT, // tone, rules, examples — a few KB, static
cache_control: { type: "ephemeral" },
},
],
tools: TOOL_DEFINITIONS, // also static, also worth caching
messages: conversation,
});
The mechanics: a cache write costs roughly 1.25× the base input rate, a cache hit costs roughly a tenth of it. On a bot where the system prompt and tools dwarf the actual user message in size (ours did, easily, once the tone-of-voice block and the booking tool schema were in there), that's most of the fixed cost of every call collapsed to a rounding error. The catch is the five-minute TTL: it resets on every hit, so it's built for the "back and forth within a conversation" pattern and does nothing for a lead who messages once and comes back tomorrow. Cache what's static, not what's fresh, and don't expect it to save a cold start.
Lever 2: stop resending turns nobody needs verbatim
Caching makes the static prefix cheap. It does nothing about the part that keeps genuinely growing: the conversation itself. Turn twenty still sends nineteen prior turns as live, uncached input, because they're not a stable prefix, they're the thing that's different every single call.
The fix isn't clever, it's a window. Keep the last N turns verbatim, because recency is where the model actually needs precision, and fold everything older into a short running summary generated once, not on every call:
function buildContext(history: Turn[]): Turn[] {
const RECENT = 6;
if (history.length <= RECENT) return history;
const older = history.slice(0, -RECENT);
const recent = history.slice(-RECENT);
return [{ role: "user", content: `Earlier in this conversation: ${summarize(older)}` }, ...recent];
}
summarize() is one cheap Haiku call, run once when the window first overflows and cached (in your own store, not Anthropic's) against the turn count, not on every message. A twenty-turn conversation now sends a summary plus six turns, not twenty turns, and that shape stays roughly flat no matter how long the lead keeps talking. This is the lever that turns the cost curve from "grows with the conversation" into "grows with a constant", and it's the one most tutorials skip because it doesn't show up until you have real, long conversations to look at.
Lever 3: route by task, not by default
I've written about model tiering before (the WhatsApp bot's Problem 3): Haiku on the shared key, a stronger model only when a tenant brings their own. That's the coarse version. The finer one is realizing a single conversation turn isn't one task, it's several, and they don't all deserve the same model.
Answering the lead is one call. Deciding whether this message even needs a tool call or whether the lead just qualified themselves is a much smaller, much more mechanical classification task riding along on the same turn. Running both through the same model with the same max_tokens is paying reasoning-model prices for a task that's closer to a regex with taste:
const classification = await anthropic.messages.create({
model: "claude-haiku-4-5",
max_tokens: 20, // it's picking one of four labels, not writing an essay
system: "Classify this message as: booking, question, qualified_lead, or other. Reply with one word.",
messages: [{ role: "user", content: lastMessage }],
});
max_tokens: 20 isn't a typo, it's the actual budget the task needs, and it's doing real work: it caps the output-token cost of a call that has no business generating paragraphs, and it caps how long a misbehaving model can ramble before the API cuts it off. The reply-writing call still gets 400 tokens and the good model. The classification call gets twenty tokens and the cheap one. Multiply that gap by every message that passes through a qualification pass, and it's a bigger number than choosing Haiku over Sonnet once and calling it done.
Lever 4: a kill switch, because a bug is not a cost, it's a cost multiplier
Every lever so far assumes the code is working correctly. The WhatsApp bot's Problem 4 covered a tool-use loop that could, if unbounded, keep calling the model until the serverless function got killed. I framed that as a latency and reliability bug. It's also, undersold, a budget bug: an iteration cap of five stops a runaway loop from becoming a runaway invoice, not just a slow reply.
That's a ceiling on one call. The one that actually saved sleep is a ceiling on a tenant's whole day, checked before any call gets made:
async function withinBudget(tenantId: string): Promise<boolean> {
const spentToday = await getTodaySpend(tenantId); // sum of usage * rate, from your own log
return spentToday < DAILY_CAP_USD;
}
if (!(await withinBudget(tenantId))) {
return safeFallbackReply(); // "we'll get back to you shortly" beats a silent 500
}
The spend itself comes straight off the response you're already getting back, no separate accounting system required:
const { usage } = message;
const cost =
usage.input_tokens * RATE.in +
usage.output_tokens * RATE.out +
(usage.cache_creation_input_tokens ?? 0) * RATE.cacheWrite +
(usage.cache_read_input_tokens ?? 0) * RATE.cacheRead;
Log that per call, per tenant, and two things fall out for free: the daily cap above, and a dashboard that answers "which tenant is expensive and why" without guessing. On a shared key serving multiple tenants, that answer used to be a mystery. Now it's a GROUP BY.
Lever 5: batch what doesn't need to be instant
Not everything is a WhatsApp reply waiting on a human. Adorea's editorial-calendar drafts and Dairector's script-doctor passes don't need a response in two seconds, they need a response by the time someone checks back in an hour. Anthropic's Message Batches API takes exactly that shape of workload, a pile of independent requests with no user staring at a spinner, and runs it at roughly half the per-token price:
const batch = await anthropic.messages.batches.create({
requests: drafts.map((d) => ({
custom_id: d.id,
params: { model: "claude-haiku-4-5", max_tokens: 800, messages: d.messages },
})),
});
You poll for completion instead of holding a connection open, which is a small code change for anything that was already running as a background job. The rule of thumb that made this easy to apply: if a human isn't waiting on the response right now, it doesn't belong on the interactive path, and it definitely doesn't belong paying the interactive-path price.
What actually moved the number
Roughly in order of leverage, for a bot with real conversation depth: caching the static prefix, then windowing the history, then right-sizing max_tokens per task, then the budget cap (which saves nothing on a normal day and everything on a bad one), then batching whatever isn't real-time. The first two alone took the cost of a long conversation from growing with its length to growing with a near-flat constant, which was the actual bug — not that Claude was expensive, but that I was paying for the same context on repeat and hadn't noticed.
None of these are exotic. They're the kind of thing that looks obvious in a blog post and invisible in a usage dashboard until you go looking, which is exactly why the token bill was the last piece of the weekend AI-product stack I got around to writing up. The demo never has a long conversation. Production always does.
