Privacy-preserving training techniques

What it does

It reduces what can be learned about an individual from a trained model, addressing memorisation at its source instead of filtering it afterwards. The model owner is accountable; the data protection officer advises on when it is required. It is decided at design and applied in every training run on sensitive data.

How to implement it

• Minimise first: remove or pseudonymise personal data that the task does not need.
• Deduplicate training data, since repeated records are memorised most readily.
• Apply regularisation and stop training early to limit overfitting.
• Use differentially private training where the sensitivity of the data justifies the cost to accuracy.
• Test the trained model with membership inference and extraction attempts.
• Record the techniques used and the test results in the model documentation.

Evidence it operates

Design: a documented decision on which techniques apply, linked to the data protection impact assessment. Operation: training configuration showing the techniques in use; privacy test results for the released model; the gap between training and test performance within a set tolerance.

Where it fits

Applied during model development and checked at validation. It is the primary control for membership and attribute inference and model inversion and training data extraction.

Limits

These techniques trade privacy against accuracy, and strong guarantees can make a model noticeably worse. They are unavailable for a model bought already trained. They reduce what can be recovered and do not make a model trained on personal data free of data protection obligations. Their effect should be demonstrated by testing, not assumed from the name of the technique.