Training

Training is the process that turns a machine learning approach into a working model. The program is shown a large collection of examples (its training data) and its internal numbers (its weights) are adjusted over and over until it gets good at the task. When people say a model was "trained on" something, they mean that material was the examples it learned from. A model can be trained from scratch, or an existing model can be further trained (fine-tuned) on a smaller, more specific set of examples to adapt it to a particular task or language.

Two things about training matter most for language work. It is data-hungry: a model is only as good as the examples it saw, so for a low-resource language with little digital text the result is often weak, and the temptation is to gather more data by whatever means (see scraping). And training bakes the data in: whatever is in the examples, including errors, gaps, stereotypes, or material taken without permission, is absorbed into the model and is hard to remove afterward. That is why what a model is trained on, and whose data it is, is a data sovereignty and consent question and not only a technical one.

Updated
Supported By the National Science Foundation Award 2542375.