Dataset documentation

What it does

It makes a dataset understandable to people who did not assemble it. A standard record, kept with the data, states what it contains, where it came from, whose data it includes, what it may and may not be used for, and what is known to be wrong with it. The data owner is accountable for the record. It is created when a dataset is first prepared for AI use and updated at every material change.

How to implement it

• Adopt a standard template, such as a datasheet, scaled to the dataset's importance.
• Record source, collection method, time period, population covered and labelling process.
• Record rights, licence, lawful basis and the purposes the data may be used for.
• Record known gaps, biases and quality results.
• Record which models and systems use the dataset.
• Version the record with the data and review it on a set cycle.

Evidence it operates

Design: a documentation standard and template. Operation: a current record for each dataset in use; version history matching changes to the data; records referred to in validation and in impact assessments; links from the inventory.

Where it fits

Alongside the data throughout its life. With lawful basis review it is a primary control for unlawful or undocumented training data basis, and it supports purpose creep in AI data use and unfulfillable data subject rights by recording what data is where. It supplies the data section of model and system documentation.

Limits

A record is only as accurate as its last update, and data changes more often than documents. It describes what its authors knew. For bought datasets and pre-trained models the detail is often unavailable, and the record should say so plainly. Documentation makes misuse recognisable and does nothing to prevent it. Where a dataset is copied and adapted by several teams, each copy needs its own record or the original soon describes nothing in use.