We Costed an AI Audit System Two Ways. The Fine-Tuned Model Won, But Not for the Reason Everyone Says.

A real cost breakdown for building document-heavy AI systems in the UK and Europe, with September 2026 prices.

It is a incredibly common question in engineering leadership right now.

“Should we fine-tune our own model, or just call the API?”

The question is almost always triggered by a company’s first massive OpenAI or Anthropic bill — and the mild panic that follows. Naturally, teams expect the answer to be, “Fine-tune it, it’s cheaper.” But while that is sometimes right, it’s usually for much narrower reasons than people think. For many teams, it’s flatly wrong.

So instead of arguing about it in the abstract, I costed a real system both ways.

First, a thing that changed and nobody told you

Before the numbers, one piece of housekeeping that quietly reshaped this whole debate.

You cannot fine-tune a current frontier model.

OpenAI’s own pricing page now says it is winding down the fine-tuning platform. It is closed to new users, and the only model still listed for tuning is o4-mini at $100 per training hour. Anthropic has never offered fine-tuning through its own API. The one path that ever existed was Claude 3 Haiku supervised fine-tuning on Amazon Bedrock, and Haiku 3 is now retired outside Bedrock and Google Cloud.

So when someone says “fine-tune versus frontier” in late 2026, what they actually mean is:

Take an open-weight model — Qwen, Llama, a Nemotron, gpt-oss — train a LoRA adapter on your data, and serve it yourself or through a host like Fireworks or Together.

That is a different decision with a different risk profile. Worth being precise about it before you put it in a board deck.

The system we costed

Not a toy. A full audit-evidence platform for a mid-size firm, the kind of thing a UK or Nordic accounting practice actually buys.

Here is what it does, end to end:

1. Intake. Documents land from the client portal. Invoices, purchase orders, goods-received notes, bank statements, lease contracts, board minutes. Mostly scanned PDFs.
2. Classification. Each document gets a type and gets routed to the right test.
3. Extraction. Pull structured fields into a fixed schema. Vendor, date, net, VAT, PO reference, approver, currency, cost centre. Output must be valid JSON that matches the schema exactly, every time.
4. Control testing. Three-way match. Does the invoice agree with the PO and the goods-received note? Is the approver inside the delegated authority limit? Most of this is plain code, not a model.
5. Exception triage. Anything that fails a test gets scored and ranked.
6. Workpaper drafting. The system writes the first draft of the finding. “We tested 42 payments above the £50,000 threshold. Three lacked documented secondary approval.”
7. Auditor Q&A. The auditor asks questions across the whole engagement file and gets grounded answers with citations back to source documents.

Steps 2 and 3 are where almost all the tokens go. They are also the most boring, most repetitive, most schema-bound parts of the system. Steps 6 and 7 are where the judgement lives, and they are a rounding error on volume.

That split is the entire story. Hold onto it.

Where the tokens go

For costing, assume documents are OCR’d to text before a model sees them. That avoids vision-token pricing and it is what you would do anyway.

Per document, across three calls (classify, extract, self-check):

• ~5,200 input tokens — the document, the schema, a few worked examples
• ~600 output tokens — the JSON

Two volumes:

• Pilot: 100,000 documents a month. One firm, one busy season.
• Scale: 2,000,000 documents a month. The same product sold to fifty firms.

That works out to 580 million tokens a month at pilot, and 11.6 billion at scale.

The numbers

List prices, standard tier, global routing, no batch or cache discounts. All verified against the vendors’ own pricing pages in September 2026.

At pilot volume, the whole argument collapses. Running the entire extraction pipeline on Claude Sonnet 5 costs $1,640 a month. On Haiku 4.5, $820. The fine-tuned 8B model costs $116.

You will not recover the cost of building a fine-tune by saving $1,500 a month. Not even close. Ship on the API.

At scale volume, it starts to look like a real decision:

• Claude Opus 5: $82,000/month
• GPT-5.6 Sol: $65,600/month
• Claude Sonnet 5: $32,800/month
• Claude Haiku 4.5: $16,400/month
• GPT-5.6 Luna: $3,520/month
• Fine-tuned 8B, serverless: $2,320/month

The headline everybody wants to write is “fine-tuning saved 93%.” That number is real, and it compares the fine-tune to Opus 5. But you should never have been running bulk invoice extraction on Opus 5. Comparing your optimised architecture against someone else’s worst architecture is not analysis, it is marketing.

The uncomfortable finding

Look at the bottom two rows again.

GPT-5.6 Luna: $3,520. Fine-tuned 8B: $2,320.

The gap is $1,200 a month. Fourteen thousand dollars a year.

Building that fine-tune properly means labelling data, writing an eval harness, running training, standing up serving, and wiring monitoring for drift. Call it eight engineer-weeks. Whatever your blended European contract rate is, multiply it out. At most rates I have seen, you are looking at somewhere north of eighteen months to pay that back on token savings alone.

So here is the honest version:

The biggest single cost win is not fine-tuning. It is routing.

Moving classification and extraction off the flagship and onto the cheapest competent tier takes Opus-5 economics down from $82,000 to $3,520 a month. That is a 96% reduction, and you can do it in an afternoon with a router and a decent eval set. The fine-tune, on top of that, buys you another 34%.

The training cost, for what it’s worth, is almost free. Fireworks charges $0.50 per million training tokens for LoRA supervised fine-tuning on models up to 16B. Twenty thousand labelled examples at 2,500 tokens each, three epochs, is 150 million training tokens. Seventy-five dollars a run. Eight runs during development is $600.

The GPU time was never the expensive part. The data and the evaluation are. Anyone quoting you a fine-tuning project based on compute cost has not built one.

So why fine-tune at all?

Two reasons. Neither is the token bill.

Reason one: schema adherence

This is the part that matters for audit specifically, and it is badly under-discussed.

A frontier model is a generalist. Ask it to return one of 51 fixed subcategories and it will occasionally invent a 52nd, because inventing plausible things is what it was trained to do. In a chatbot that is a minor annoyance. In a workpaper that feeds a signed audit opinion, it is a defect.

There is a 2026 preprint on security document classification that measured exactly this. On a benchmark with a fixed 51-subcategory taxonomy and a strict JSON output protocol, a fine-tuned local model beat prompted frontier models by 15–20 percentage points. The authors observed GPT-5 inventing subcategory names that did not exist in the taxonomy. Their explanation is the interesting part: on loosely-formatted extraction tasks, frontier models do fine. Under strict schema constraints, calibration beats reasoning.

Another preprint found a fine-tuned Qwen2.5–0.5B — half a billion parameters, runs on a single consumer GPU — scoring 0.83 average micro-F1 on general-domain relation extraction against 0.69 for GPT-5.4 and 0.66 for Claude Sonnet 4.6 under a minimal zero-shot protocol. The authors are careful to say this does not make small models intrinsically stronger. It shows that targeted task adaptation is worth more than raw capability on a narrow, fixed task. That is exactly what audit extraction is.

Anthropic published a similar result from their own fine-tuning work on Claude 3 Haiku: classification accuracy went from 81.5% to 99.6% on a comment-moderation task, with 85% fewer tokens per query. Different domain, same shape of finding.

For an audit product, a two-point accuracy gain on field extraction is worth more than the entire inference bill. Every miss becomes a human review. Human review is the expensive thing in this business.

Reason two: you know where the data is

This is the one that decides deals in Europe.

The EU residency problem

If your client is a UK or EU accounting firm, their audit files contain client financial data, employee data, and sometimes personal data belonging to third parties. “Where does this run” is not a nice-to-have question. It is the second thing procurement asks.

Here is the current state of play, and it surprises people:

Anthropic’s first-party API does not offer EU data residency. The inference_geo parameter accepts global or us. US-only costs 1.1x. There is no EU option. If you need Claude inside the EU, you route through AWS Bedrock or Google Vertex in an EU region, and regional endpoints carry a 10% premium over global. Claude on Microsoft Foundry still has no EU data zone as of writing.

OpenAI does offer regional processing, at a 10% uplift for models released on or after 5 March 2026.

Fireworks charges a 1.5x premium for region-restricted dedicated deployments.

Your own GPUs in a Frankfurt or Dublin data centre cost whatever they cost, and the answer to “where does the data go” is “nowhere.”

Run the scale-volume numbers again with residency priced in:

Two dedicated H100s running your fine-tuned 8B model, region-restricted, twenty-four hours a day, comes to about $17,520 a month. Claude Sonnet 5 through a Bedrock EU endpoint at the same volume is $36,080. Opus 5 is $90,200.

That is the real EU case for owning your model. Not that it is cheaper per token. That the compliance story is a sentence instead of a diagram.

But don’t buy GPUs too early

The flat-rate GPU line is seductive and it traps people. It only wins above a volume most products never reach.

Against two dedicated H100s at global pricing ($11,680/month):

• Claude Sonnet 5 stays cheaper below ~700,000 documents a month
• Claude Haiku 4.5 stays cheaper below ~1.4 million
• GPT-5.6 Luna stays cheaper below ~6.6 million

Most audit products will spend their first two years to the left of all three of those lines. Serverless hosting of your own fine-tuned model sidesteps the problem entirely — you get the custom weights without the idle GPU bill — which is why it is where I would start every time.

The decision framework

Strip out the noise and it comes down to four questions.

1. What is your monthly token volume, honestly? Below roughly a billion tokens a month, use the cheapest competent frontier tier and move on. You have better places to spend eight engineer-weeks. Fine-tuning at pilot volume is engineering theatre.

2. Is the task narrow and schema-bound? Fixed output format, fixed label set, thousands of near-identical examples a day. That is a fine-tune. Open-ended reasoning, judgement calls, drafting a narrative for a partner to sign — that is a frontier model, and it always will be.

3. Where does the data have to live? If a contract clause says “processing must remain in the EEA,” that changes the shortlist before cost enters the conversation. Price the residency premium at the start, not after the architecture review.

4. Do you have 10,000 labelled examples, or a way to get them? NVIDIA Research’s position paper on small models in agentic systems puts the rule of thumb at 10,000 to 100,000 examples for fine-tuning a small model. If you do not have that, your first project is not a fine-tune. It is instrumenting your frontier-model pipeline so it logs its own training data.

That last point deserves emphasis, because it is the pattern I recommend most often and it is the one nobody markets.

Ship on the frontier API. Log every call. Six months later you have a labelled dataset that cost you nothing, and you fine-tune from a position of knowledge instead of a position of hope.

The same paper estimates how much of three real open-source agent systems could move to small models: about 60% of LLM calls in MetaGPT, 40% in Open Operator, 70% in Cradle. Not 100%. Not zero. The answer is a mix, and you cannot guess which calls without the logs.

One counter-argument, stated fairly

Menlo Ventures’ enterprise survey found open-source models falling to 11% of enterprise workloads, down from 19%. While everyone was writing about how small models are the future, enterprises were consolidating onto closed frontier APIs. Anthropic took 40% of enterprise LLM spend, OpenAI 27%, Google 21%.

That is not a contradiction of everything above. It is a measurement of where the friction is. Closed APIs win on convenience, and convenience usually wins. The teams getting real value from open-weight fine-tunes are the ones who deliberately took on operational burden for a reason they could articulate. If you cannot articulate the reason, the survey is telling you something.

On the regulation, briefly

A quick note since UK and EU readers will ask.

The EU’s Digital Omnibus on AI — Regulation (EU) 2026/1744 — was published in the Official Journal on 24 July 2026 and entered into force on 27 July. It pushed high-risk obligations for standalone Annex III systems from 2 August 2026 out to 2 December 2027, and for AI embedded in regulated products under Annex I to 2 August 2028.

What did not move: the Article 50 transparency obligations, which applied from 2 August 2026 as originally scheduled. Article 50(2) marking for systems already on the market applies from 2 December 2026.

An audit-support tool with a human sign-off gate generally does not land in Annex III. But check your own use case rather than taking my word for it, because if any part of your system touches hiring decisions or creditworthiness, it does. And none of this touches GDPR, which applies to the client data in your pipeline no matter how the AI Act classifies the system.

Demand for this is not theoretical, by the way. A Wolters Kluwer survey of 4,214 internal audit professionals found 39% already using AI and another 41% planning to adopt within twelve months.

What I’d actually tell a client

If you came to us with this build, here is the shape of the answer.

Phase one. Ship it on the cheapest frontier tier that passes your eval set. Instrument every model call. Get it in front of real auditors during a real busy season. Do not fine-tune anything.

Phase two. Read the logs. Find the two or three calls that account for most of your tokens. Check whether they are schema-bound and repetitive. They almost always are.

Phase three. Fine-tune those calls, and only those. Serve them serverless first. Keep the frontier model for triage, drafting and Q&A. Route between them.

Phase four. Move to dedicated GPUs only when volume or a residency clause forces it. Not before.

Four phases, and three of them are cheaper than the thing most people do on day one.

The question was never “small model or frontier model.” It was “which of my calls actually need to think.” Answer that and the cost question answers itself.

Sources

All figures verified September 2026. Prices change frequently — check them before you commit a budget.

• Anthropic, model pricing — platform.claude.com/docs/en/about-claude/pricing
• OpenAI, API pricing and fine-tuning wind-down — developers.openai.com/api/docs/pricing
• Fireworks AI, serverless and training pricing — fireworks.ai/pricing and docs.fireworks.ai/serverless/pricing
• Belcak et al., Small Language Models are the Future of Agentic AI, NVIDIA Research, arXiv:2506.02153
• Security Document Classification with a Fine-Tuned Local Large Language Model, arXiv:2605.20368 (preprint)
• Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs, arXiv:2606.22606 (preprint)
• Anthropic, Fine-tune Claude 3 Haiku in Amazon Bedrock — claude.com/blog/fine-tune-claude-3-haiku
• Menlo Ventures, 2025 State of Generative AI in the Enterprise (Menlo is an investor in Anthropic)
• Regulation (EU) 2026/1744 (Digital Omnibus on AI); EU AI Act implementation timeline, European Commission
• Wolters Kluwer internal audit AI survey, 4,214 respondents