What is fine-tuning?

What is fine-tuning and how does it work?

You start from a base model that a provider or an open-weights project already trained, and you continue training it on a dataset of examples that look like the task you actually want it to do. Each example is typically a prompt and the completion you would have wanted the model to produce. The trainer computes the loss between what the model would say and what you wanted, then nudges the weights to close that gap, one small step at a time. After enough steps the resulting checkpoint answers your kind of prompt more reliably than the base model did, and you deploy that checkpoint as if it were any other model.

Two things follow from the mechanics. First, the change is baked into the weights: at inference the prompt can be shorter, because behaviour you used to spell out is now the model's default. Second, the model has no idea which examples were in the training set; it has generalised from them, which is what makes it useful and also what makes it fail on inputs unlike the ones you provided. A fine-tuned model is not a lookup table over your data.

Providers expose this as an API. OpenAI's supervised fine-tuning takes a JSONL file of chat-shaped examples, runs a training job, and returns a model ID you can call with the ordinary completions endpoint. The shape of your data is usually the largest lever on the outcome, ahead of the hyperparameters.

Full fine-tuning, LoRA and other PEFT methods

Full fine-tuning updates every weight in the model. For anything but a small model this is expensive to train and expensive to serve, because you now hold a full-size checkpoint per task. Parameter-efficient fine-tuning, or PEFT, is the family of methods that avoids that by freezing the base weights and training a much smaller set of additional parameters that ride alongside them.

LoRA is the most common member of that family. Rather than editing a weight matrix directly, it learns a pair of low-rank matrices whose product is added to it, which for a typical transformer means training a fraction of a percent of the parameters and shipping an adapter of a few megabytes instead of a full checkpoint. The LoRA paper reports matching full fine-tuning quality on several tasks while cutting the trainable parameter count by orders of magnitude, and Hugging Face's PEFT library packages LoRA and its relatives behind a common interface.

Practically, LoRA adapters are cheap enough that you can keep several around per base model and swap them per request. Full fine-tuning is worth the money only when the adapter approach measurably underperforms on your eval.

Fine-tuning vs prompting vs RAG

These solve different problems and are often confused because all three change what the model outputs. A better prompt changes the instructions the model reads on every call, costs nothing to iterate on, and is the right first move for tone, format and simple behaviour. RAG changes the facts the model sees by retrieving passages at query time, and is the right move when the answer depends on documents that update. Fine-tuning changes the model itself, and is the right move when a behaviour is stable, hard to describe in a prompt, and repeated often enough that the training cost pays back.

The order matters. If you fine-tune before you have exhausted prompting, you bake in the shape of a prompt you have not finished designing, and every future change now costs a training run. If you fine-tune on facts that will change next quarter, you have moved a retrieval problem into your weights, where it is much harder to update. Reach for fine-tuning when the prompt is long, the format is strict, or the base model keeps drifting on a task you can demonstrate but not concisely explain.

What breaks a fine-tune

The failure modes are consistent across providers. Too few examples and the model overfits to their surface form, repeating phrases from the training set on unrelated inputs. Inconsistent labels teach the model that both answers are acceptable, which shows up as a regression in reliability rather than a clean error. Training data that drifts from your production distribution produces a model that scores well on your holdout and worse than the base model in the wild.

Two operational habits catch most of this. Keep an eval set the trainer never sees, and run it against the base model as well as each candidate checkpoint, so you can tell whether the fine-tune actually beat prompting. Version your training data alongside your code, because a fine-tune is only reproducible if the JSONL that produced it is.

# training file: sql-assistant.jsonl (1 line per example)
{"messages":[
  {"role":"system","content":"You write PostgreSQL. Reply with SQL only."},
  {"role":"user","content":"top 5 customers by revenue last quarter"},
  {"role":"assistant","content":"SELECT customer_id, SUM(amount) AS rev\nFROM orders\nWHERE ordered_at >= date_trunc('quarter', now()) - interval '3 months'\n  AND ordered_at <  date_trunc('quarter', now())\nGROUP BY customer_id\nORDER BY rev DESC\nLIMIT 5;"}
]}
# ... 800 more lines like this, drawn from real tickets

# submit the job
curl https://api.openai.com/v1/fine_tuning/jobs \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{"training_file":"file-abc123","model":"gpt-4.1-mini-2025-04-14"}'

# result: a new model id you call like any other
# ft:gpt-4.1-mini-2025-04-14:acme::9x2Kp
#   prompt shrinks from ~600 tokens of instructions to ~40