Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Fine-tuning

Fine-tuning is further training of a pretrained model on a smaller, task-specific dataset, so the model adopts a particular skill, format, tone or domain without being trained from scratch.

Also called: finetuning

By Divya Sharma · Checked Nov 16, 2026 · How we review

A pretrained model has learned general language from a very large corpus. Fine-tuning takes those weights and keeps training them on examples you choose: support tickets answered the way your team answers them, code in your house style, or medical notes in a fixed format. The training loop is the same as pretraining; the difference is that you start from a working model rather than random weights, so it needs far less data and compute.

Kinds of fine-tuning

Supervised fine-tuning trains on input and desired output pairs. Preference tuning, such as RLHF or DPO, trains on which of two answers people preferred. Full fine-tuning updates every weight; parameter-efficient methods such as LoRA train a small add-on and leave the base model frozen, which cuts memory and lets you keep many adapters for one base model.

When to use it

Fine-tuning is good at teaching behaviour: a consistent output format, a narrow classification task, a house voice, or making a small model do one job as well as a larger one. It is a poor way to add facts that change often, because the knowledge is frozen at training time and hard to inspect. For that, retrieval-augmented generation is usually the better first step, and many teams try careful prompting before either.

Limits

You need clean, representative examples, an evaluation set held back from training, and a plan to retrain when the base model is updated. A model can also forget general skills if it is trained too hard on a narrow set. The fine-tuning stage guide lists tools such as Unsloth and Axolotl.

Related terms

LoRA
LoRA (low-rank adaptation) is a fine-tuning method that freezes a model's original weights and trains small low-rank matrices added to its layers, so far fewer parameters change and much less GPU memory is needed.
Supervised fine-tuning
Supervised fine-tuning (SFT) trains a pretrained model further on example prompts paired with the responses you want. It is the usual first step that turns a base model into one that follows instructions.
RLHF
RLHF (reinforcement learning from human feedback) fine-tunes a language model using human rankings of its answers: a reward model learns those preferences, then reinforcement learning pushes the model toward answers that score well.
Direct preference optimization
Direct preference optimization (DPO) is a way to align a language model with human preferences by training directly on pairs of preferred and rejected answers, using a simple classification-style loss instead of a reward model and reinforcement learning.
Pretraining
Pretraining is the first and largest stage of training a model, where it learns general patterns from a very large unlabelled dataset, usually by predicting the next token, before any task-specific tuning.
Parameter-efficient fine-tuning
Parameter-efficient fine-tuning (PEFT) adapts a pretrained model by training only a small number of new or selected parameters, leaving the rest frozen. It cuts the memory, compute and storage that full fine-tuning needs.

Sources