There's a quiet principle in yoga that applies surprisingly well to technology: use exactly the effort the moment requires — no more, no less. Most teams running AI today are doing the opposite. They're spending maximum effort on every task, whether it needs it or not, and paying for the privilege one token at a time.
Let's talk about why your AI costs keep climbing — and how to bring them back to something intentional.
Frontier models are built to consume tokens
The frontier models from Anthropic and OpenAI are genuinely remarkable, and a large part of why is that the newest generation simply does more work per request. They reason step by step. They "think" before they answer. They run longer, multi-step agentic loops. They give thorough, well-hedged, context-rich responses.
This is a real user-experience win. Answers are better, more careful, and more capable than the terse models of a couple of years ago. But it comes with a structural side effect worth naming plainly:
These models are priced per token, and they are designed to use more tokens.
More reasoning, more output, more back-and-forth — all of it improves the experience and increases consumption. The incentives of the people selling you tokens and the incentives of you, the buyer, are not pointed in the same direction. You end up paying for deep deliberation on tasks that needed a one-line answer.
What this actually costs a business
On a single chat, the cost is invisible — a fraction of a cent. The problem is scale.
- A customer-support workflow that handles 50,000 messages a day.
- A content pipeline generating hundreds of drafts.
- An internal agent that loops five or six times per task to "reason it through."
Multiply per-call token usage by reasoning overhead, then by volume, then by 30 days. A workflow that looked trivially cheap in the demo becomes a serious, recurring operating expense — and one that grows every time you add a feature or a customer. Worse, it's unpredictable: a model that decides to think harder this month quietly raises your bill without anyone shipping a single line of new code.
For a small or mid-sized business, that unpredictability is the real tax.
The local alternative: Ollama
Here's the shift in mindset. Not every task needs a frontier model. In fact, most don't.
Ollama lets you run capable open-source models — Llama, Mistral, Qwen, and many others — directly on your own hardware or a modest server. Once it's running, the marginal cost of a request drops to roughly zero. You've already paid for the compute; there's no per-token meter spinning.
The trade-off is that a local 7B–30B model isn't as broadly brilliant as a frontier model. But for well-defined, repeatable work — classification, extraction, summarization, routing, drafting against a template — a smaller local model is more than good enough. The art is knowing which tasks to keep local and which to escalate.
AnythingLLM's custom skills: the real lever
This is where most of the savings live.
AnythingLLM is an open-source application that wraps local models (via Ollama) in a clean workspace with document/RAG support — and, crucially, a custom skills system. Instead of asking a model to reason its way through the same task from scratch every single time, you encode the repeatable part of the job as a deterministic skill: real code that runs predictably, calls your APIs, formats your output, and only hands the genuinely ambiguous decisions back to the model.
The result is a hybrid:
- Deterministic code does the heavy, repeatable lifting — for free, every time, identically.
- The model is invoked only for the small slice of the task that actually requires judgment.
You get the same end result, with a fraction of the token consumption — because you stopped paying a language model to re-derive a process you already know.
Where MCP servers fit in
A small local model is only as useful as what it can reach. That's the role of MCP (Model Context Protocol) servers.
MCP is an open standard for connecting AI models to tools, data, and live systems — databases, file stores, internal APIs, ticketing systems, your CRM. AnythingLLM supports MCP servers, which means your local stack isn't a sealed chatbot; it's an agent that can read real data and take real actions through a clean, standardized interface.
This is what makes the local approach viable for serious work. The frontier providers want you to do everything inside their token-metered context window. MCP lets you keep the data, the tools, and the deterministic logic on your side — and only spend tokens on the actual thinking.
Practical tips: rewriting a standard skill as an optimized custom skill
Here's how to take a task you currently throw at a frontier model and rebuild it to run for a fraction of the cost.
Profile your token spend first. Find the 3–5 workflows that run most often. Those are where savings compound. Don't optimize the thing you run twice a week.
Separate the deterministic from the ambiguous. Read through what the model actually does on each call. The parts that are the same every time — fetching data, applying a format, validating fields, routing — don't need a language model at all.
Move the deterministic parts into code. In an AnythingLLM custom skill, that logic becomes a handler function. It runs identically every time, with zero token cost.
Shrink the prompt to the decision. Hand the model only the genuinely ambiguous question, with just enough context to answer it. No restating the whole process every call. Smaller input, smaller output, smaller bill.
Pull live data through MCP, not the prompt. Don't paste a 4,000-token document into context when an MCP tool can fetch exactly the field you need on demand.
Route by difficulty. Run the everyday version on a local Ollama model. Reserve the frontier model for the genuinely hard 5% — and let your skill decide when to escalate.
Measure the before and after. Track tokens-per-task before and after. The teams that do this routinely find 80–95% reductions on their highest-volume workflows.
The intentional bottom line
Frontier AI is worth paying for — when the task truly demands it. The mistake is using it as a default for everything, and quietly funding a per-token meter that's designed to keep climbing.
Run the routine work locally. Encode what's repeatable as custom skills. Connect it to the real world with MCP. Save the frontier models for the moments that genuinely need them.
That's not cutting corners. That's just using the right effort for the moment — which, as it happens, is good practice on the mat and in the cloud.