Personal data redaction and pseudonymisation

What it does

It reduces the personal data a model and its logs contain. Before data enters a training set, a test set or a log, identifying details are removed or replaced, and where data is to be treated as anonymous the strength of the anonymisation is tested. The data owner decides what may remain; data engineering applies and tests the technique. It operates in the data pipeline before each use and continuously on logging.

How to implement it

• Decide for each use whether identifiable data is needed at all.
• Remove direct identifiers and replace them with consistent pseudonyms where linkage is required.
• Treat free text separately, with detection of names, addresses and other details.
• Address indirect identifiers by generalising or suppressing rare combinations.
• Keep any re-identification key separately and restrict access to it.
• Test by attempting re-identification, and record the result.

Evidence it operates

Design: a standard stating which techniques apply to which data and uses. Operation: pipeline records showing the step ran; sampled outputs checked for residual identifiers; re-identification test results; access records for keys.

Where it fits

In the data pipeline, after provenance and before training and logging. It is the primary control for re-identification of anonymised data and, with privacy-preserving training, for training data memorisation and regurgitation and model inversion and training data extraction.

Limits

Pseudonymised data is still personal data, and the duties continue to apply. Automated detection misses identifiers in free text, and unusual records stay identifiable by their content. Stronger anonymisation removes detail a model may need, so there is a real trade against usefulness. Anonymity is judged against the other data available, which changes over time.