Unfulfillable data subject rights

What it is

People have rights over their personal data: to see it, correct it, have it erased and object to its use. AI systems hold that data in places conventional procedures do not reach: training sets copied long ago, embeddings in a vector store, prompt and conversation logs, cached outputs, and the model's own parameters. A request that would take minutes against a database may be impossible to satisfy fully against these. The risk is that the system was built with no thought for how a request would be answered.

How it happens

• Training data was assembled with no record of whose data it contains.
• Embeddings cannot be traced back to the documents and people they came from.
• Logs are held by a provider with no search by individual.
• Removing one person's influence from a trained model is not practical.

Why it matters

Failure to respond properly to a rights request is a breach, and such requests are a common trigger for regulatory complaints. Where data cannot be found or removed, the organisation may face a choice between non-compliance and rebuilding.

Warning signs

• The rights procedure does not mention AI systems.
• Nobody can say whether a named person's data is in a training set.
• Vector stores keep no link to source records.
• An erasure request has been closed with model data left untouched and unexplained.

Who owns it

The data protection officer owns the rights procedure. Each system owner is accountable for being able to answer for their system. Engineering builds the means to locate and remove data.

In practice

A former customer asks for erasure. Their account is deleted. Their correspondence remains in a fine-tuning dataset, in the vector index built from it and in two years of conversation logs at the provider. The same pattern generalises to any system whose data stores were designed without the question of how one person's data would be found.