Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

AI training data

AI training data is the collection of examples a model learns from, such as web text, code, images, transcripts or labelled pairs. Its size, quality, mix and licensing shape what the model can do and where it fails.

Also called: training set, dataset

By Divya Sharma · Checked Nov 16, 2026 · How we review

Every model reflects its data. A large language model is pretrained on a very large corpus of text and code, then fine-tuned on much smaller sets of instructions, demonstrations and preference pairs. An image model learns from images paired with captions. Gaps or biases in the data show up as gaps or biases in the model, and anything not in the data before the knowledge cutoff is unknown to it.

Kinds of training data

Pretraining data is mostly collected at scale, for example by web crawling, then filtered, de-duplicated and mixed. Fine-tuning data is smaller and curated, often written or ranked by people. Synthetic data generated by other models is now common in both. Evaluation sets, such as a golden dataset, must stay separate from training data, or scores become meaningless through benchmark contamination.

Documentation and rights

Gebru et al.'s "Datasheets for Datasets" proposes that every dataset come with a datasheet describing its motivation, composition, collection process and recommended uses. A model card often summarises the data a model was trained on. Copyright, consent, licences and personal data are live legal questions in many places; rules such as the EU AI Act add documentation duties. This page does not give legal advice.

For builders

If you fine-tune, a few thousand clean, representative examples usually beat a large noisy set. Our AI training data directory lists dataset sources and vendors.

Related terms

Synthetic data
Synthetic data is training or test data generated by software, often by another AI model, rather than collected from real-world sources. It is used to fill gaps, add examples cheaply, or avoid exposing real personal data.
Pretraining
Pretraining is the first and largest stage of training a model, where it learns general patterns from a very large unlabelled dataset, usually by predicting the next token, before any task-specific tuning.
Fine-tuning
Fine-tuning is further training of a pretrained model on a smaller, task-specific dataset, so the model adopts a particular skill, format, tone or domain without being trained from scratch.
Web crawling for AI
Web crawling for AI is the automated fetching of web pages, by following links or a list of URLs, to collect text for training models, building search indexes or giving an application fresh content to retrieve.
Model card
A model card is a short document published with a machine learning model that explains what it is for, how it was trained and evaluated, where it performs poorly, and how it should and should not be used.
Golden dataset
A golden dataset is a fixed, carefully checked set of example inputs paired with the answers or behaviour you expect, used as the reference when you evaluate or regression-test an AI system.

Sources