For Heads-of · Practitioner
Model Collapse
The degradation in quality and diversity that occurs when language models are trained, generation after generation, on content generated by other language models.
- model collapse
- training data
- synthetic data
- degradation
Mechanism
As AI-generated content makes up more of the available training data, new models learn the artefacts and blind spots of earlier models rather than the patterns of real human data. Rare cases disappear first, and quality and diversity decline with each generation.
Risk amplification
Errors in early models are inherited and amplified by later ones, which can produce systematic failures across a whole ecosystem of models trained on each other's output.
Long-term governance concern
Relying on synthetic data for training needs a check on where the data came from and how much of it is real. Training data provenance checks are the control, and the degradation they guard against is a form of AI model drift and degradation.
Related Terms
Data Quality
The degree to which data is accurate, complete, consistent and representative of what it is meant to measure.
Model Drift
A change in the relationship between a model's inputs and the environment it was fitted to, such that the model's assumptions no longer hold. The model itself is unchanged; the world it was trained on has moved.
Synthetic Data
Artificially generated data, created by algorithms or models, used instead of or alongside real data in training and testing.
Training Data
The dataset used to train a machine learning model, from which the model learns patterns and relationships.