Fine-Tuning vs Prompt Engineering: Cost Math (2026)

TL;DR: Fine-tuning a small open-weight model now starts at $0.34 per million training tokens (Together AI, Llama 3.1 8B) with a $4 minimum job — and at roughly $0.000295 per request for inference, a fine-tuned small model breaks even against a $2/$10-per-million flagship at just ~600 requests. But serve that fine-tuned model on a dedicated H100 ($5.49/hr ≈ $3,953/month) and the break-even leaps to ~590K requests/month — below that volume, prompt engineering wins outright. Meanwhile, OpenAI is winding down its fine-tuning platform (existing users only; o4-mini at $100/hour), and Anthropic offers no self-serve fine-tuning at all. Training is a one-time cost; the inference and hosting choices decide everything. All prices checked 21 October 2026.

Every team running an LLM workload eventually asks the same question: should we fine-tune a model, or keep investing in better prompts? The honest answer is a math problem, not an opinion — and the 2026 numbers make it sharper than most people expect. Training got cheap enough that the training bill barely matters. What matters is where the fine-tuned model runs afterward, and which providers even let you fine-tune anymore.

The headline change this year: OpenAI is winding down its fine-tuning platform. Their pricing page now carries a notice — read today, 21 October 2026 — that the platform "is no longer accessible to new users," though existing users can keep creating jobs for the coming months. If you were planning your fine-tuning strategy around OpenAI, that plan just expired for new teams.

What fine-tuning actually costs in 2026#

Training is priced per million training tokens (dataset size × epochs, plus evaluation tokens), and the spread across providers is enormous. All prices below checked 21 October 2026.

Open-weight models on Together AI (official pricing page, read today):

Fireworks AI, banded by model size (read 18 September 2026): up to 16B parameters costs $0.50/M training tokens for LoRA supervised fine-tuning; 16.1–80B costs $3.00/M; 80–300B costs $6.00/M; over 300B costs $10.00/M.

Google Vertex AI supervised tuning (official rate card): Gemini 2.5 Flash Lite $1.50/M, Gemini 2.5 Flash $5/M, Gemini 2.5 Pro $25/M, Gemma 3 27B $6.83/M, Llama 3.1 8B $0.67/M. Note that the consumer Gemini API shut down fine-tuning on 27 May 2025 — Vertex is the only Google path. Tuned 2.5-generation endpoints predict at the base model's price; from Gemini 3 onward, tuned endpoints cost 1.5× the base rate.

OpenAI: winding down (verified on the official pricing page today). The one remaining published option is o4-mini-2025-04-16: $100 per hour of training (billed hourly, not per token), with inference on the fine-tuned model at $4.00/$16.00 per million input/output tokens — or $2.00/$8.00 if you enable data sharing. Existing users only.

Anthropic: no self-serve fine-tuning on the Claude API at all — custom training exists only through enterprise agreements. AWS Bedrock does offer fine-tuning for Claude 3.5 Haiku (Claude 3 Haiku reached end of life on 10 September 2026).

DeepSeek: no managed fine-tuning on the platform API. The weights are MIT-licensed and fine-tunable yourself; the hosted API is inference-only.

To put training in perspective, here is what 10 million training tokens cost on each option — about a 20,000-example dataset run for 3 epochs:

USD→BDT at ≈৳123.5 per $1 (mid-market, 21 October 2026).

The point that matters: training is a one-time cost, and on the cheap end it is the price of a coffee. Training is not what decides the business case. Hosting is.

Fine-tuned models on Together AI deploy to dedicated endpoints billed hourly — an H100 at $5.49/hour runs about $3,953/month (≈৳488,171), charged continuously even with zero traffic. Fireworks AI's Multi-LoRA approach is the alternative shape: fine-tuned adapters served serverless at per-token rates, with no fixed hosting bill. Which of these two shapes you land on changes the break-even volume by a factor of roughly a thousand. That is the whole story of this post.

What prompt engineering actually costs#

Prompt engineering has no invoice line, which is why teams underprice it. It has three real costs:

1. Engineering time. Dataset curation for fine-tuning also costs time, so call this one roughly a wash for the math — but prompt work never really ends, because every base-model update can shift behavior and every edge case sends you back to the prompt.

2. The token tax on every request. This is the one you can put a number on. Prompt engineering works by making prompts longer: few-shot examples, detailed instructions, format specifications. Those tokens bill on every single request, forever.

   A 500-token system prompt at GPT-6 Sol's $2.00 per million input tokens costs $0.001 per request — invisible per call, but at 1 million requests a month it is $1,000/month (≈৳123,500) of pure instruction overhead. A fine-tuned model internalizes those instructions into weights, so that tax drops to zero. Killing the prompt tax is usually the single largest recurring saving fine-tuning buys.

3. Fragility. A prompt is a contract written in natural language. Fine-tuning is a behavior baked into weights. Neither is permanent — base models get replaced either way — but a fine-tuned small model tends to degrade more gracefully on its trained task than a long prompt does on a swapped model.

The break-even math (worked example)#

Stated assumptions, so you can swap in your own numbers:

• One request = 1,000 input tokens + 500 output tokens.
• Option A (prompt engineering): a flagship-class model at GPT-6 Sol's published rates, $2.00/$10.00 per million. Cost per request: 0.001 × $2.00 + 0.0005 × $10.00 = $0.007.
• Option B (fine-tune, serverless): Qwen3.5 9B on Together AI. Training: 3M tokens × $0.34/M = $1.02, raised by the $4.00 minimum to $4.00 one-time. Inference at the model's listed serverless rate ($0.17/$0.25 per million, Multi-LoRA style): 0.001 × $0.17 + 0.0005 × $0.25 = $0.000295/request.
• Option C (fine-tune, dedicated): same as B, plus a dedicated H100 at $5.49/hour × 24 × 30 = $3,952.80/month.

The formula: break-even requests = fixed costs ÷ (big-model cost per request − small-model cost per request). Per-request saving: $0.007 − $0.000295 = $0.006705.

• A vs B: $4.00 ÷ $0.006705 ≈ 597 requests. Under 600 requests and the fine-tuned model has already paid for its training.
• A vs C: $3,952.80 ÷ $0.006705 ≈ 589,530 requests/month. Below ~590K requests a month, the dedicated hosting alone costs more than the entire prompt-engineering bill.

Prices checked 21 October 2026. USD→BDT at ≈৳123.5.

At 100K requests a month, the serverless fine-tuned model saves $670.50/month (≈৳82,807) over the flagship — a 95%+ cost cut, with training paid back in the first hour of traffic. But notice what the table really says: options B and C are the same training run with different deployment economics. The $4 training job is irrelevant in both; the hosting decision moves the break-even by ~1,000×. Anyone who tells you "fine-tuning breaks even at X requests" without naming the hosting model is not doing the math.

Two more honest caveats before you act on these numbers:

• Quality is assumed, not guaranteed. Every figure above assumes the fine-tuned small model matches the big model on your task. That is an evaluation question, not a pricing question. If it doesn't match, no amount of cost math saves the project — run your own evals first.
• The new-model treadmill. Fine-tuned weights are pinned to a base model. When the next generation ships (and it ships every few months now), you retrain. Prompt engineering ports forward for free. This is a real, recurring cost of the fine-tuning path.

When fine-tuning wins, and when it doesn't#

The most expensive mistake in this decision, seen over and over: reaching for training to solve a retrieval problem. Fine-tuning does not reliably add new facts — it teaches behavior, style, and format. If the model needs current prices, policies, or product details, that is retrieval-augmented generation, not a training job. Training on last quarter's data bakes last quarter's answers into the weights, and fixing them means retraining. As Anthropic's own guidance puts it: a strong system prompt, few-shot examples, and prompt caching come first; training is the last resort. When the lab that builds the model treats fine-tuning as a last resort, that is a strong signal for where it belongs in your own ladder.

Fine-tuning providers compared#

Provider landscape checked 21 October 2026:

Where does tokens.bd fit? Tokens does not offer fine-tuning itself — and you should be suspicious of any gateway that claims to. What the 60+ model catalog gives you is inference access to the fine-tunable class of base models — DeepSeek V4.1 Flash, Kimi K2.6, GLM-5.2, Qwen 3.8 27B — at per-token prices, so you can prototype against a candidate base model before committing to a training run on the provider's own platform. Compare the gateway route against direct APIs in our OpenRouter comparison, and check the docs for calling any model through one OpenAI-compatible endpoint.

Code: a fine-tuning job and the inference call after it#

# Train on Together AI: LoRA supervised fine-tuning of Llama 3.1 8B
from together import Together

client = Together()
job = client.fine_tuning.create(
    training_file="train.jsonl",   # one {"messages": [...]} JSON object per line
    model="meta-llama/Llama-3.1-8B-Instruct",
    training_type={"type": "Lora"},
    n_epochs=3,
)
print(job.id)
# Cost: (dataset tokens x 3 epochs) x $0.34 / 1M tokens, $4.00 minimum per job

# Serve it: the fine-tuned model answers through the same API shape
from openai import OpenAI

client = OpenAI(base_url="https://api.together.xyz/v1")
r = client.chat.completions.create(
    model="your-org/ft-llama-3.1-8b-support-v1",  # your fine-tuned model id
    messages=[{"role": "user", "content": "Where is my order #4821?"}],
)
print(r.choices[0].message.content)

The workflow is the same on every provider: JSONL examples in, a model id out, inference through the standard chat endpoint. The part that differs — and the part to price first — is how that model id is hosted.

Related reading#

• Model routing pairs naturally with fine-tuning: route the easy 70% of traffic to your fine-tuned small model and escalate only the hard cases to a frontier model.
• How your LLM API bill really works breaks down the five line items (input, output, cache reads, cache writes, reasoning tokens) that the break-even math above is built on.
• Prompt caching explained — if long system prompts are your main cost, caching can cut them 90%+ without any training at all.

Fine-tuning vs prompt engineering FAQ#

How much does it cost to fine-tune a model in 2026? Training runs from $4 minimum jobs (Together AI, models up to ~70B parameters) to $150–$400 for 10M training tokens on large models like Kimi K2.6 or GLM-5.2. Most practical runs land between $5 and $150 one-time. The dominant cost is not training — it is hosting the fine-tuned model afterward, which can run ~$3,953/month on a dedicated H100 or near-zero on serverless Multi-LoRA.

Can I still fine-tune on OpenAI? Only if you are an existing fine-tuning user. OpenAI is winding down the platform: it is closed to new users, existing users can create jobs for the coming months, and the only model with a published training price is o4-mini-2025-04-16 at $100/hour, with inference at $4/$16 per million tokens ($2/$8 with data sharing). Do not start a new fine-tuning project on OpenAI.

Can I fine-tune Claude? Not self-serve. The Claude API offers no fine-tuning product; custom training exists only through enterprise agreements. AWS Bedrock offers fine-tuning for Claude 3.5 Haiku (Claude 3 Haiku reached end of life on 10 September 2026). Anthropic's own guidance points teams toward system prompts, few-shot examples, and prompt caching instead.

At what request volume does fine-tuning break even? It depends on hosting, not training. In the worked example: with serverless per-token inference, break-even against a $2/$10 flagship is ~600 requests; with dedicated H100 hosting at $5.49/hour, it is ~590K requests/month. Use the formula — fixed costs ÷ per-request savings — with your own rates.

Will fine-tuning teach the model new facts? No, not reliably. Fine-tuning teaches behavior, style, format, and task-specific patterns. For knowledge that changes — prices, policies, product data — use retrieval-augmented generation. Training on facts bakes in a snapshot that goes stale and can only be fixed by retraining.

All provider prices checked 21 October 2026. Fine-tuning products and rate cards change — the provider's pricing page is the authority, and the model catalog for prices on Tokens.

Sources: OpenAI pricing (fine-tuning wind-down notice) · Together AI pricing · Google Vertex AI pricing · CloudZero on Vertex AI pricing · Fine-tuning cost comparison (Together + Fireworks tables) · Fine-tuning vs RAG vs prompting · OpenAI vs Anthropic vs Gemini API 2026 · DeepSeek V4.1 Flash pricing · DeepSeek V4 Pro review · Gemini pricing (API tuning shutdown) · Fine-tuning Claude 3 Haiku on Bedrock · Together AI review (serverless + dedicated rates) · Together AI pricing guide · Fine-tuning cost guide · OpenAI API pricing 2026