Step 15 of 22
Fine-tuning
Adapt an open model to your data, tone or task. Fine-tuning fixes style and narrow skills cheaply. It is a poor way to add facts; that is retrieval's job.
How to choose
- One GPU and a small budget: LoRA tooling built for efficiency.
- Repeatable, multi-GPU training: config-driven frameworks.
- Image styles and characters: LoRA trainers for diffusion models.
What to look for
The questions that separate the options, in order of how much they matter.
01
LoRA or QLoRA first, full fine-tuning only with a reason
LoRA trains small adapter weights on top of a frozen model; QLoRA does the same on a model loaded in 4-bit, so it fits in far less memory. Full fine-tuning updates every weight and is worth it only when the model's behaviour must shift a long way. Try LoRA first: it is cheaper, quicker to iterate on, and the adapter can be swapped in and out at serving time. Managed services such as Together AI make LoRA the default for the same reason.
02
GPU memory, worked out before you rent
Memory decides which GPU, and how many, you need. LlamaFactory's published estimates for a 7B model are a useful rule of thumb: about 120 GB for full fine-tuning in mixed precision, 60 GB in pure bf16, 16 GB for LoRA and 6 GB for 4-bit QLoRA. Scale roughly with parameter count, and add headroom for long sequences. Vendor speed and VRAM claims, such as Unsloth's 2x faster with 70% less memory, are best treated as upper bounds that vary by model.
03
The training methods you will actually need
SFT (supervised fine-tuning on example answers) covers most style and format work. Preference methods such as DPO, KTO and ORPO teach the model which of two answers is better. RL methods such as GRPO reward correct answers on checkable tasks like maths or code. Check that the tool supports the method you need now and the one you are likely to need next: LlamaFactory, for example, sends GRPO users to a separate project, while TRL, Axolotl and Unsloth support it directly.
04
Single GPU or many
Single-GPU efficiency and multi-GPU scale are different problems. Unsloth is built around squeezing a run onto one card. Axolotl treats FSDP2, DeepSpeed, multi-node and sequence parallelism as first-class, which matters once a model or context no longer fits on one GPU. TRL inherits DDP, DeepSpeed and FSDP from Accelerate. If you expect to grow past one GPU, pick a tool whose distributed path is documented rather than an afterthought.
05
Config, code or UI
YAML-driven tools (Axolotl, LlamaFactory) make runs reproducible and easy to share; many model cards publish the exact Axolotl config. Libraries (TRL, Transformers) give full control of the training loop for research or unusual pipelines, at the cost of more code. UIs (LlamaFactory's LlamaBoard, Unsloth Studio, AI Toolkit) lower the barrier for experiments. Choose by who will rerun the job in six months.
06
Model coverage and how fast new models arrive
A tool is only useful if it supports the base model you want, including its chat template and any vision parts. Transformers usually gets new architectures first and the training tools follow; LlamaFactory lists over 100 language and vision-language models. Check the changelog for how quickly the last few major open models were supported, not just the current list.
07
Export path to your serving stack
Training is half the job. Check that the result can be merged and exported in the format your inference engine loads: Safetensors for vLLM and SGLang, GGUF for llama.cpp and Ollama. Unsloth exports GGUF directly; managed services such as Together AI let you download the weights or serve them on a dedicated endpoint. A run you cannot serve cheaply is wasted.
08
Licence of the tool and of the base model
Most frameworks here are Apache 2.0 or MIT, but Unsloth's optional Studio UI is AGPL-3.0, which matters if you embed it in a product. The bigger constraint is usually the base model: its licence carries over to your fine-tune, including any use restrictions or naming rules.
How the options differ
The options fall into four families. Training libraries (TRL, Transformers) are Python code you assemble yourself; they get new methods first and give full control, but you own memory tuning and distributed setup. Recipe frameworks (Axolotl, LlamaFactory) wrap those libraries in YAML configs or a web UI, so a run is one file and one command; Axolotl leans towards multi-GPU scale, LlamaFactory towards breadth of models and a no-code UI. Efficiency-first tools (Unsloth) rewrite GPU kernels to fit bigger models on one card.
Two outliers complete the picture. AI Toolkit trains LoRAs for image and video diffusion models, a different job from tuning a language model. Managed fine-tuning APIs such as Together AI's remove the GPUs entirely: you upload data, pay per training token and get an endpoint or downloadable weights, at the cost of fewer methods (GRPO, for example, sits outside its managed service) and less control.
7 tools, compared
Ranked by our rating. Open any review for sources, prices and the full verdict.
| Tool | Rating | Our verdict | Best for | Price |
|---|---|---|---|---|
| Hugging Face Transformersrepo | 4.8/5 | Nearly every open-model tool reads its model code from Transformers, which makes it essential to know and rarely the thing you serve with. | Python developers and researchers working with open models at any level. | Free and open source under the Apache 2.0 licence. Hugging Face sells Hub plans and hosted inference separately. |
| Unslothrepo | 4.6/5 | Unsloth turns a single 24 GB card into a credible fine-tuning rig, and stops being the obvious choice at the second server. | Individuals and small teams fine-tuning open models on limited GPU hardware. | Free. Core library Apache 2.0; Studio UI AGPL-3.0. Paid Pro and Enterprise tiers exist for multi-GPU and multi-node training; prices are not published. |
| TRLrepo | 4.5/5 | TRL is the post-training code that friendlier fine-tuning tools wrap; go to it directly when the training loop itself is what you need to change. | ML engineers and researchers building custom post-training pipelines. | Free and open source under the Apache 2.0 license. |
| Together AItool | 4.5/5 | A single bill for open models at every stage, from a $15 API top-up to a reserved H100 cluster, at prices that undercut the specialist hosts on dedicated hardware. | Teams building on open models who want to move from API calls to fine-tuning and their own GPUs with one provider. | Prepaid, $15 minimum first purchase, no free trial. Per 1M tokens, e.g. gpt-oss-120B $0.15 in / $0.60 out, Kimi K3 $3 / $15; batch up to 50% off. Dedicated endpoints: H100 $5.49/hour, B200 $8.99/hour. GPU clusters: H100 $3.99/hour on demand, $1.99 preemptible. Fine-tuning from $0.34 per 1M tokens. |
| Axolotlrepo | 4.3/5 | Axolotl is where fine-tuning goes once one GPU stops being enough: one YAML file, every parallelism strategy, and a learning curve to match. | Practitioners running repeatable fine-tunes, especially across several GPUs. | Free and open source under the Apache 2.0 license. |
| LlamaFactoryrepo | 4.3/5 | LlamaFactory turns fine-tuning into a web form across 100+ open models, which suits experiments better than a pinned production pipeline. | People who want to fine-tune many different open models through a graphical interface. | Free and open source under the Apache 2.0 license. |
| AI Toolkitrepo | 4.2/5 | AI Toolkit is where support for a new open image or video model usually lands first, and it expects you to bring the GPU and the patience. | Creators training LoRAs for open image and video models. | Free and open source under MIT. Ostris Cloud rents preconfigured GPUs by the second, from $0.71/hour (A40 48 GB) to $2.99/hour (RTX PRO 6000 96 GB). |
Side by side
The specifications that matter for this step. Checked 2026-10-19; confirm on the vendor’s site before you commit.
| Tool | Interface | Methods | LoRA / QLoRA / full | Multi-GPU | Hardware | Licence |
|---|---|---|---|---|---|---|
| Hugging Face Transformers | Python library (Trainer) | Supervised training | Full; LoRA via PEFT | DDP, DeepSpeed, FSDP | PyTorch devices | Apache 2.0 |
| Unsloth | Python library, Studio UI, desktop | SFT, DPO, GRPO, GSPO, pretraining | LoRA, QLoRA, full, FP8 | Supported (setup in docs) | NVIDIA, AMD, Intel GPUs | Apache 2.0 core; Studio AGPL-3.0 |
| TRL | Python library + CLI | SFT, DPO, KTO, GRPO, reward | LoRA/QLoRA via PEFT, full | DDP, DeepSpeed, FSDP (Accelerate) | PyTorch GPUs, AWS Trainium | Apache 2.0 |
| Together AI | Managed API | SFT, DPO (GRPO via separate product) | LoRA (default), full | Managed | Together's GPUs | Paid, per training token |
| Axolotl | YAML config + CLI | SFT, DPO, KTO, ORPO, GRPO, reward | LoRA, QLoRA, full, QAT | FSDP2, DeepSpeed, multi-node | NVIDIA (Ampere+), AMD | Apache 2.0 |
| LlamaFactory | Web UI + YAML CLI | SFT, RM, PPO, DPO, KTO, ORPO, SimPO | LoRA, QLoRA 2–8-bit, freeze, full | DeepSpeed, FSDP, Megatron | NVIDIA, AMD, Ascend NPU | Apache 2.0 |
| AI Toolkit | YAML config + web UI | Diffusion LoRA training | LoRA (image and video models) | Not published | High-end GPU typical | MIT |
Where this stage fits
- Image generation pipeline: we would start with AI Toolkit.
- Your own fine-tuned model: we would start with Unsloth, Axolotl.