Training Data

Training data is the raw material that AI models learn from. Just like a student needs textbooks to study, an AI model needs data to learn from. For a language model, training data might include billions of web pages, books, articles, and code repositories. For an image model, it might be millions of images paired with descriptions.

The quality and diversity of training data has a huge impact on how good the resulting AI is. If you train an image recognition model only on photos of golden retrievers, it will be useless at identifying poodles. If a language model is trained mostly on English text, it will struggle with other languages. And if the training data contains biases -- for example, if it over-represents certain viewpoints -- the model will learn and repeat those biases.

Collecting and preparing training data is one of the hardest parts of building AI systems. Companies spend enormous effort gathering data, cleaning it up (removing duplicates, offensive content, and errors), and organizing it for training. There are also thorny legal and ethical questions: much of the text and images used to train today's AI models were scraped from the internet without explicit permission from the original creators, which has led to lawsuits and heated debates.

When people say an AI model has a "knowledge cutoff," they mean the model only knows things that were in its training data. If the training data only goes up to a certain date, the model will not know about events after that date unless it has access to a search tool or other way to fetch current information.