Built this week · I didn't know about that
Cheaper AI isn't always cheaper
I moved my daily Claude brief to a cheaper model. At Anthropic's listed prices it cost between 22% and 71% more a run, and it told me things that weren't true. One morning it said "No new security alert" with one sitting in the mail it had just read. It also wasn't the version that had been tested, and that went unnoticed for more than two weeks.
Why I tried it
Around 5 am most mornings, Claude goes through my mail and my task board and tells me what needs me that day. I act on what it says.
It ran on Opus, one of Claude's stronger models. Sonnet is a step down. AI use is measured and priced in tokens, and Sonnet's price for the tokens it writes is half of Opus's. If you pay for Claude, you've probably wondered the same thing I did: does this job really need one of the stronger models?
How I tested it
Claude ran a one-day test of Sonnet 5.5 first, and it looked good. So I gave Sonnet a week, seven briefs. Claude counted what every run used and compared them with Opus runs from before and after. Separate Claude helpers, running on Opus, then checked the seven Sonnet briefs and two of the Opus ones against what each had actually read.
What came back
Replies a run, against Opus's earlier runs
Cost a run, against Opus's earlier runs
Cost a run, against Opus's later runs
Sonnet took more than twice as many replies, and every reply re-reads everything so far. Its prices were no higher, but it used far more. I can't say how much of the gap is the model. It ran at a higher effort setting than planned, and my brief kept growing, so neither Opus comparison is exactly like for like.
The slips worried me more. Figures it copied straight from a tool were right. These weren't:
| What the brief said | What was actually true |
|---|---|
| "No new security alert." | A Google security alert had come in the evening before. It was my own phone, but the brief should have told me. |
| "The fortnightly review due today ... was task 132" | Task 132 was a different job. The review ran the next day. |
| "Done: 90 of 95 tasks" | It had read the first 100 of about 356 tasks. |
What went wrong, and what it cost
Finding those slips took a full review, which cost US$12.04 at the same listed prices. It also checked two Opus briefs and counted the costs. Checking costs money too, whichever model you're on.
Most of the slips never got a fix of their own. I moved back to Opus on 28 October and didn't change the instructions for them, so some could come back.
The week also didn't run on the version that had been tested. The one-day test was Sonnet 5.5. The week ran on Sonnet 5, which costs twice as much to re-read, and I still don't know why. The first brief that week even said it couldn't confirm which version it was, and nobody picked that up until Claude's own check of this article's figures. If it used the same tokens, the same run at Sonnet 5.5's prices would be about US$2.52, about 21% above Opus's earlier runs and about 14% below its later ones. So Sonnet 5.5 hasn't had a fair go, and it may well do this job fine.
What I learnt
It really is about the right model for the right job. The price per token isn't the price of the job, and one day's test didn't show what the week did. An answer can read well and still be wrong, and going back to where it came from is how you spot that.
This is only one job, on my set-up, so I'm not saying Sonnet's a bad model. Opus missed something too, a task timer that had been left running. A lighter model may be all you need for everyday writing and sorting, and this brief reads through a whole business, which is a harder job.
What it could mean for you
If Claude's doing real work in your business, it's probably going well. It's saving you time and nobody's complaining. That's how my brief looked too.
So here's what I'd ask about any job you've come to rely on:
- Which model is it actually running on?
- When did someone last check its answers against where they came from?
- How much does the whole job use, start to finish?
If you can answer those, you're ahead of where I was. If you want to try a cheaper model, give it a week on real work and check what it tells you.
If something's not sitting right, I can look at it with you. My Fix-up goes through how your Claude is set up and used, including the job itself, the Claude version you're on and your usage limits. Then I run the job three times on a copy of a past example. You get a plain report on what's going wrong and why. If nothing's wrong on the day, it says so. It won't catch every one-off slip, so you'll still want someone checking what goes out.
See the figures behind this article
You can check the prices and the run sums from here. The run records hold my business mail, so I can't publish them. The models are Opus 5.5 and, for the week, Sonnet 5. In Australian dollars, Sonnet 5 was about A$5.08 a run against A$2.97 for Opus's earlier runs.
One average run
Tokens are how AI use gets measured and priced.
| Type of token | Opus 5.5, earlier runs | Sonnet 5 |
|---|---|---|
| Re-read | 2,776,791 at US$0.20 a million = US$0.56 | 10,533,150 at US$0.20 a million = US$2.11 |
| Stored for re-reading | 154,459 at US$8 a million = US$1.24 | 263,134 at US$4 a million = US$1.05 |
| Written | 14,700 at US$20 a million = US$0.29 | 41,021 at US$10 a million = US$0.41 |
| Total | US$2.09 | US$3.57 |
Over all the runs, Opus's nine earlier runs total US$18.77 and Sonnet 5's seven total US$24.99. Opus's nine later runs, 28 October to 6 November, total US$26.44, or US$2.94 a run, at 21.2 replies a run against Sonnet's 48.0 and 18.4 for the earlier runs. The same Sonnet tokens at Sonnet 5.5's re-read price of US$0.10 a million come to US$2.52 a run, though Sonnet 5.5 might not use the same tokens. Sonnet ran at a higher effort setting than planned, and the brief grew during its week, so none of this is exactly like for like.
What each mistake took
| Mistake | What it took | Cost | Could it still be there? |
|---|---|---|---|
| Sonnet's slips, six kinds | No fix. I moved back to Opus. | US$0 | Yes. Later briefs show no repeat so far, but they haven't been checked against their sources, and the instructions weren't changed. |
| Mail search only looking in the inbox | Instructions changed | Not counted | Looks fixed. None of the later searches did it. |
| Newsletters missed. My instructions were at fault. | A repair, then a check | US$14.93 | No |
| Opus missed a task timer left running | Hours and counts corrected | US$19.03 | The brief now lists open timers on finished tasks. |
Those costs take in the work and the wrap-up around it. The newsletter repair is US$5.42 for the time on the task, plus US$9.51 for the check. The task timer is US$4.51, US$6.03, US$3.43 and US$5.06. The US$12.04 review had 3 helpers, and the records undercount what helpers write, so it may be a little more.
Every run
17 and 18 October ran on a different model, Fable, so they're left out.
| Date | Model | Replies | Tokens stored | Tokens re-read | Tokens written | US$ |
|---|---|---|---|---|---|---|
| 10 Oct | Opus 5.5 | 15 | 146,244 | 2,073,952 | 14,178 | 1.87 |
| 11 Oct | Opus 5.5 | 16 | 153,809 | 2,140,458 | 14,494 | 1.95 |
| 12 Oct | Opus 5.5 | 15 | 146,169 | 2,087,076 | 13,350 | 1.85 |
| 13 Oct | Opus 5.5 | 17 | 173,248 | 2,604,354 | 14,415 | 2.20 |
| 14 Oct | Opus 5.5 | 23 | 120,806 | 2,938,791 | 14,163 | 1.84 |
| 15 Oct | Opus 5.5 | 21 | 139,630 | 3,145,075 | 16,368 | 2.07 |
| 16 Oct | Opus 5.5 | 21 | 153,825 | 3,478,996 | 13,062 | 2.19 |
| 19 Oct | Opus 5.5 | 18 | 173,751 | 3,080,847 | 16,382 | 2.33 |
| 20 Oct | Opus 5.5 | 20 | 182,647 | 3,441,571 | 15,887 | 2.47 |
| 21 Oct | Sonnet 5 | 45 | 254,751 | 9,655,450 | 43,493 | 3.39 |
| 22 Oct | Sonnet 5 | 44 | 311,880 | 11,068,053 | 36,149 | 3.82 |
| 23 Oct | Sonnet 5 | 47 | 276,001 | 10,419,659 | 44,627 | 3.63 |
| 24 Oct | Sonnet 5 | 41 | 265,227 | 8,735,211 | 44,370 | 3.25 |
| 25 Oct | Sonnet 5 | 53 | 235,850 | 10,678,304 | 37,601 | 3.46 |
| 26 Oct | Sonnet 5 | 59 | 215,526 | 12,132,863 | 35,721 | 3.65 |
| 27 Oct | Sonnet 5 | 47 | 282,702 | 11,042,509 | 45,184 | 3.79 |
The mistakes on record
These are the ones the 28 October review listed. Three had been caught before it. It checked each brief against what that brief had read, so anything in mail it never read wouldn't show. It also listed a 4.5 h figure on 24 October that couldn't be confirmed either way, so that's left out.
| Brief | Mistake | Caught | Fix on record | Still there? |
|---|---|---|---|---|
| Sonnet, 21 Oct | Newsletter issues missed, from a fault in my instructions | Same day | Repair and a check, US$14.93 | No. Later briefs count them. |
| Sonnet, 22 and 26 Oct | Mail search only looking in the inbox | 23 and 28 Oct | Instructions changed, cost not counted | Looks fixed. None of the later searches did it, but Opus was back by then. |
| Sonnet, 22 Oct | "0 new Queensland or federal law", with the federal register's timeout only mentioned just above it | 28 Oct | None | Could be. The federal register hasn't timed out again. |
| Sonnet, 23 Oct | Fortnightly review shown as done | 28 Oct | None | Not seen since. |
| Sonnet, 23 Oct | "No new security alert" with one in its mail | 28 Oct | None | Later briefs not checked against the mail. |
| Sonnet, 26 Oct | "ends 5:30 am (4:30 am GMT+11)". The times were swapped. | 28 Oct | None | Later briefs have it right. |
| Sonnet, 27 Oct | Counts, inbox and commits from part of a list | 28 Oct | None. The instructions still don't say to read the whole list. | Could be. Later counts look right. |
| Sonnet, 21 to 26 Oct | Times and days wrong, and a stale item carried over | 28 Oct | None | Weekdays right from 28 Oct on. The rest not seen since. |
| Opus, 20 Oct | "Clocks: none found running from an earlier day". One had been running since 4:52 am on 19 Oct. | Same day, by the one-day Sonnet test | Hours and counts corrected, US$19.03 | The brief now lists open timers on finished tasks. |
Sources
- Anthropic, "Pricing" (Opus 5.5 and Sonnet 5.5 read on 27 October and 6 November 2026, Sonnet 5 read on 6 November 2026): https://platform.claude.com/docs/en/about-claude/pricing
- Reserve Bank of Australia, table F11.1, 27 October 2026 rate, US$0.7021 to A$1: https://www.rba.gov.au/statistics/tables/
- Prompt Works Studio's own run records and briefs, 10 October to 6 November 2026 (private).
Newsletter
A short email from me now and then about AI changes that matter to small businesses.
Sign up to the newsletter