Hey Everyone,
Last issue I told you I was actively working on the next video. This is it.
It is about the least glamorous decision in your entire AI stack:
Who decides which model runs, and who owns the bill when that decision goes wrong?
That sounds boring until your product is full of hardcoded model names, your team starts running longer agent loops, and your AI bill quietly turns into a recurring mystery charge.
So I put the video right up front this time.
Watch this first
Own the Bill: Why Your AI Stack Needs a Model Router
If you only skim this email, watch that.
The visual version matters here because "model router" can sound like infrastructure homework. In the video I walk through the actual flow: request comes in, policy picks a route, the verifier checks the answer, and the system escalates when the cheap path is not good enough.
That is the whole idea:
Stop treating the model name like the strategy. Make the strategy explicit.
Why I Made This One
Everybody wants to talk about which model is smartest this week.
That is useful, but it is not enough.
The thing that actually determined whether AI teams had a good month recently was whether their architecture could survive a model disappearing.
Because that happened. Anthropic pulled Fable 5 - arguably the best coding model available - and it was gone for three weeks. Their writeup on redeploying it is worth reading on its own. But the more interesting part was watching teams realize, all at once, that their "AI architecture" was actually one hardcoded model name copy-pasted across every workflow, prompt, agent, and internal app they run.
The outage got the headlines.
The overpaying is quieter, and it happens every month.
If every support summary, enrichment job, and routine classification in your business defaults to the most expensive model you can buy, you do not have a cost problem. You have an ownership problem.
Brian Armstrong posted about Coinbase cutting AI token costs while usage went up. That is not magic. That is routing.
The Short Version
A model router is the layer between your app and the models.
It turns model choice into policy:
Cheap path first for work that should not need premium reasoning.
Quality checks before accepting the cheap answer.
Escalation when the output misses the bar.
Receipts so you can see what ran, what it cost, and why it escalated.
Not "always use the cheapest model."
Not "add another abstraction because architecture diagrams look nice."
The useful version is: route, verify, escalate, and keep the receipt.
Use This After You Watch
I made the companion guide the home base for everything:
Drop your email there and you get the free router repo:
mock CLI that runs with zero API keys
visual demo
LiteLLM gateway config
editable policy file
model-router audit skill
The repo is intentionally runnable without paying an API bill. I wanted you to be able to see the routing pattern before you wire it into anything real.
If you already use my skills marketplace, the audit skill is there too:
/plugin marketplace add Getting-Automated/skills
/plugin install model-router-audit@getting-automatedA Model Name Is a Preference. A Router Is a Policy.
Right now, most AI features are built like this:
Somewhere in the code, there is a string with a model name in it.
That string was chosen months ago, probably because it was the best model the day the feature shipped.
That string is now your architecture.
It decides your cost. It decides your failure mode. It decides what happens when the provider has a bad week.
A router flips that. Instead of every workflow saying "use this exact model," the application sends the request - plus metadata, constraints, and the expected quality bar - into a routing layer.
The router decides which model should handle the job, runs it, checks the output, and escalates when the cheap path did not clear the bar.
Four things fall out of that:
Cost control - simple work goes to cheaper models by default.
Resilience - when a model disappears, you change one policy file instead of every caller.
Quality gates - expensive models run when the verifier says they are needed.
Receipts - you can see what each route costs, which requests escalated, and where the policy is weak.
The router does not make model choice magically safe.
It gives you one place where model choice is explicit.
That distinction is the whole video.
The Policy File Is the Product
The demo repo routes with three mechanisms, in order of how much I trust them:
1. Tags
The caller says what kind of work this is - routine-summary, customer-facing, code-gen - and the policy maps tags to models.
Boring. Reliable. Start here.
2. Rules
Constraints like context size, latency budget, and data sensitivity narrow the candidate list.
A 200-page document does not get sent to a model with a small context window, no matter how cheap it is.
3. Classifier routes
A small, cheap model reads the request and picks the lane when the caller cannot tag it.
This is the clever one, which is exactly why it is last. Clever needs receipts.
In the video I run the whole flow with a mock CLI: deterministic, no API keys, so you can watch requests route, verify, and escalate without paying anyone. Then the same policy file drives a LiteLLM gateway for live provider calls.
Same policy. Mock or live.
That is the point: the policy is the product.
The models are interchangeable parts.
The Verifier Is Where the Savings Become Real
Here is the part people skip, and it is the same part they skip with agent loops:
Routing cheap without verification is not cost control.
It is quality roulette with a lower ticket price.
The savings only become real when the cheap model's output gets checked against a bar - schema valid, facts grounded, tone right, tests pass - and the request escalates to the expensive model only when the check fails.
That means most requests ride the cheap path and pass. Some fail and escalate. And every one of them leaves a receipt: what ran, what it cost, whether it cleared the bar.
If you cannot prove the cheap answer was good enough, you did not save money.
You deferred a quality problem.
Hope is not a strategy here either.
Proof is.
What I Would Route First
Do not start by routing your flagship customer-facing feature.
Start where the work is high-volume, low-stakes, and verifiable:
Internal summaries and meeting notes
Enrichment and classification jobs
Tagging, triage, and routing of tickets or emails
First-pass drafts that a human already reviews
Anything currently pointed at your most expensive model "just to be safe"
That last one is the quiet budget leak. "Just to be safe" is what a default becomes when nobody owns it.
Where a Router Does Not Belong
A router is not a replacement for the native tools you already like.
Claude Code, Codex, the agent products - their model choice is part of the product, tuned by people who watch those loops all day.
Do not proxy them for sport.
The router earns its keep in the systems you own: your workflows, your internal apps, your n8n and Make scenarios, your API-calling scripts. The places where a model name got hardcoded because it was launch week and everyone was busy.
Loops burn tokens. Routers decide whose tokens. If you took last issue's advice and started running longer agent loops, this is the other half of that conversation.
The Receipts
The question your reporting should answer is not "which model did we use?"
It is:
What did each route cost this week?
What share of requests cleared the cheap path?
What escalated, and why?
What would this month have cost if everything ran on the flagship?
That last number is the one to show whoever pays the bill. It turns "we should look into AI costs" into a line item you already fixed.
Before You Go
The full companion guide goes deeper than the video:
what a policy file looks like line by line
fallback that is actually tested
context-size constraints
where open-weight models fit as leverage instead of religion
I am already working on the next video, and the daily TikTok experiments continue.
Was this newsletter automated? Parts of the pipeline, sure. The opinions are expensive and hand-made.
- Hunter