Fine-tuning
Fine-tuning is further training of a pretrained model on a smaller, task-specific dataset, so the model adopts a particular skill, format, tone or domain without being trained from scratch.
Also called: finetuning
By Divya Sharma · Checked Nov 16, 2026 · How we review
A pretrained model has learned general language from a very large corpus. Fine-tuning takes those weights and keeps training them on examples you choose: support tickets answered the way your team answers them, code in your house style, or medical notes in a fixed format. The training loop is the same as pretraining; the difference is that you start from a working model rather than random weights, so it needs far less data and compute.
Kinds of fine-tuning
Supervised fine-tuning trains on input and desired output pairs. Preference tuning, such as RLHF or DPO, trains on which of two answers people preferred. Full fine-tuning updates every weight; parameter-efficient methods such as LoRA train a small add-on and leave the base model frozen, which cuts memory and lets you keep many adapters for one base model.
When to use it
Fine-tuning is good at teaching behaviour: a consistent output format, a narrow classification task, a house voice, or making a small model do one job as well as a larger one. It is a poor way to add facts that change often, because the knowledge is frozen at training time and hard to inspect. For that, retrieval-augmented generation is usually the better first step, and many teams try careful prompting before either.
Limits
You need clean, representative examples, an evaluation set held back from training, and a plan to retrain when the base model is updated. A model can also forget general skills if it is trained too hard on a narrow set. The fine-tuning stage guide lists tools such as Unsloth and Axolotl.
Related terms
- LoRA
- LoRA (low-rank adaptation) is a fine-tuning method that freezes a model's original weights and trains small low-rank matrices added to its layers, so far fewer parameters change and much less GPU memory is needed.
- Supervised fine-tuning
- Supervised fine-tuning (SFT) trains a pretrained model further on example prompts paired with the responses you want. It is the usual first step that turns a base model into one that follows instructions.
- RLHF
- RLHF (reinforcement learning from human feedback) fine-tunes a language model using human rankings of its answers: a reward model learns those preferences, then reinforcement learning pushes the model toward answers that score well.
- Direct preference optimization
- Direct preference optimization (DPO) is a way to align a language model with human preferences by training directly on pairs of preferred and rejected answers, using a simple classification-style loss instead of a reward model and reinforcement learning.
- Pretraining
- Pretraining is the first and largest stage of training a model, where it learns general patterns from a very large unlabelled dataset, usually by predicting the next token, before any task-specific tuning.
- Parameter-efficient fine-tuning
- Parameter-efficient fine-tuning (PEFT) adapts a pretrained model by training only a small number of new or selected parameters, leaving the rest frozen. It cuts the memory, compute and storage that full fine-tuning needs.