Token Economics for People Who Skipped the Engineering Track

What I couldn't really say

There is a moment in every vibe coded app where you decide whether to think about cost. It is usually around 11pm. You are on the third Liquid Death. The thing works. You hit Run.

Most of the time, this is fine. The job is small. The bill is twelve cents. You ship.

Sometimes the job is eighty thousand records and the model you used is the most expensive one your provider offers. By morning the bill is in the thousands and you are drafting a memo about how innovation cost variance is, as is typical with breakthrough technologies, hard to predict in the first quarter.

What I couldn't really say in the conference room is that the bill was a choice. It was the choice not to spend ninety seconds on the back-of-envelope estimate that anyone who has shipped a metered service for ten years does in their head before they hit Run. The choice not to use the cheap model for the easy part of the job. The choice not to cache the prompt prefix that was eighty percent of the tokens. The choice not to use the batch endpoint that costs half as much and ships overnight while you sleep.

None of these choices are technical. They are habits. The reason people who came up shipping production systems have them is they paid for them the first time, the way you just did.

Here is what I would have told you in private, if I could have.

1. Before you run anything

The back-of-envelope estimate takes ninety seconds.

• Pick the model. Look up the price per million tokens for input and output.
• Estimate the average prompt size and response size in tokens. Rough rule: one word is about 1.3 tokens.
• Multiply: (input_tokens × in_price) + (output_tokens × out_price). That is the cost per call.
• Multiply cost per call by the number of calls you plan to make.

If the result is more than $50, you are no longer prototyping. Treat this like a production job. If it is more than $500, get a second opinion before you hit Run.

Also: always pilot on 10 records first. Compare the output to your eye. Then 100. Then 1000. Then the full set. The number of jobs that failed at 80,000 because the prompt was wrong on a structural edge case that did not appear in the first three records is too many to count.

2. Model selection patterns

You do not need the premium model for most jobs.

• Classification, simple summarisation, extraction, language detection, basic translation. Use the cheap tier (Haiku, GPT-4o-mini, Gemini Flash). Ten to thirty times cheaper than the premium tier. The quality drop is usually invisible.
• Reasoning, nuanced summarisation, code generation, anything where you would be embarrassed by a wrong answer. Use the mid-tier (Sonnet, GPT-4o). This is the right default for most enterprise work.
• Multi-step reasoning, long-context analysis, agentic flows, the things you actually want to show off. Use the premium tier (Opus, GPT-4.1). Reserve it for the calls where the cost is worth the lift.

For repeated lookups where the answer should be deterministic and short, use embeddings instead of generation. Embeddings are cheap. Vector search is fast. If you are using a language model to ask "is this related to that," you are paying ten thousand times more than you need to.

3. Caching patterns

Three caches matter.

• Prompt prefix caching. If your prompt has a long static section (system prompt, instructions, examples, ontology), the provider can cache it on their end and charge you much less for that portion on subsequent calls. Anthropic offers this with the cache_control flag. OpenAI auto-applies it on prompts over a threshold. Use it.
• Response caching. If the same input is likely to be requested twice in a 24-hour window, cache the response in Redis or even on disk. The cost saved is the entire call.
• Semantic dedup. If you are processing user-submitted text, deduplicate semantically before you call the model. Two questions that mean the same thing should not result in two billed calls.

4. Batching patterns

If your job is not time-sensitive, use the batch endpoint.

• Anthropic and OpenAI both offer Batch APIs at roughly fifty percent off the standard rate.
• Trade-off: you submit a batch and get results within 24 hours.
• For overnight jobs (summarising the ticket queue, classifying support tickets, generating embeddings for a corpus), this is free money.

Always batch by chunks of 1000 to 5000 records, not the full set in one go. If the batch fails midway, you have not lost everything.

5. Cost guardrails

What separates a hobby project from a production system is the guardrails.

• Service account, not your personal key. Your personal key has your personal credit card. Set up a service account with a hard monthly cap.
• Hard spend limit. Most providers let you set a cap that pauses the account when hit. Set it for three times what you think the month will cost. Not thirty times.
• Per-request cost logging. Log the cost of every call in your application logs, tagged by feature or job. If you cannot answer "what does this feature cost per user per month," you have not instrumented it.
• Alerts. Set up an alert when daily spend exceeds the previous 7-day rolling average by fifty percent. Surprise spend is almost always a bug or a runaway loop.

The artifacts

Two formats. Same conclusions.

For your coding agent

Paste this into Claude Code, Cursor, or any reasoning-capable coding agent pointed at your repo. It will produce a structured report and propose concrete diffs.

You are reviewing my codebase for LLM cost and efficiency.

Walk the repo and locate every place that calls an LLM provider (Anthropic, OpenAI, Google, Bedrock, Azure OpenAI, Cohere, custom clients, abstractions). For each call site, produce a report with:

1. CHARACTERISATION
 What kind of task is this call doing? Pick one: classification, extraction, simple summarisation, reasoning, code generation, nuanced summarisation, multi-step agentic, other.

2. MODEL TIER REVIEW
 What model is being called? Could a cheaper tier (Haiku, GPT-4o-mini, Gemini Flash) handle the task at acceptable quality? If yes, propose the swap and estimate the cost reduction.

3. PROMPT PREFIX CACHING
 Is the prompt prefix (system prompt, instructions, examples) static across calls and longer than ~1024 tokens? If yes, propose adding cache_control (Anthropic) or note whether OpenAI's automatic caching should apply.

4. RESPONSE CACHING
 Is the same input likely to be sent twice within a 24-hour window? If yes, propose adding a response cache (Redis, in-memory, or persistent).

5. SEMANTIC DEDUPLICATION
 For user-submitted inputs, is there pre-call dedup of semantically similar requests? If no and the use case warrants it, propose embedding-based dedup.

6. BATCHING
 Is this call time-sensitive? If no (overnight job, async summary, classification at scale), propose using the Batch API (50% cheaper, 24hr SLA) and outline the refactor.

7. GUARDRAILS
 - Is the API key in source code, an env var, or a secret manager? Flag if hardcoded.
 - Is cost-per-call logged anywhere?
 - Is there a hard spend limit on the account?
 - Are there daily spend alerts?
 Flag any gaps.

OUTPUT
A structured report with one section per call site, plus a summary table at the top: file path, current monthly cost estimate, optimised monthly cost estimate, complexity to implement.

Then propose the three highest-impact changes as concrete code diffs.

Make conservative assumptions about traffic volume. Ask for actuals if a recommendation depends on them.

For your monitor

The one-page version. Before you hit Run on anything larger than 1000 records:

1. Cost per call estimated and written down.
2. Total cost estimated and written down. Acceptable to your boss without a memo.
3. Model is the cheapest tier that does the job.
4. Prompt prefix is using cache_control or equivalent.
5. Job is using the Batch API if time allows.
6. Piloted on 10 records, then 100. Output quality verified by a human.
7. Service account in use. Personal key is not in this codebase.
8. Hard spend limit set on the account.
9. Per-request cost logged in application logs.
10. Daily spend alert configured.

Print this. Put it next to your monitor. Or do not. We've done good work today either way.

Get the next field note below.