For Heads-of · Practitioner
Model Extraction
Also known as: model stealing · model theft
Reconstructing a model's behaviour or parameters by querying it repeatedly and learning from the answers, to copy its intellectual property or to build a surrogate for planning further attacks.
- model stealing
- IP theft
- inference API
- adversarial machine learning
Definition
An attacker sends many queries to a model's interface and uses the responses to train a copy that behaves similarly. A close copy reproduces the model's value without its cost, and lets the attacker rehearse evasion or jailbreak attacks offline before using them on the real system. NIST's adversarial machine learning taxonomy treats it as a privacy attack on the model itself.
Why it matters
For a firm that has invested in a proprietary model, extraction is intellectual property loss. For any firm, a surrogate makes other attacks cheaper and quieter. Models served through public or partner APIs, or embedded in widely distributed apps, are the most exposed.
Where to go next
This entry defines the term. The register entry model extraction and inference-API abuse sets out the controls, chief among them inference usage quotas and rate limiting and monitoring for unusual query patterns.
Sources
Related Terms
Adversarial Attack
An attempt to manipulate or deceive an AI model by deliberately crafting inputs designed to cause misclassification or unintended behaviour.
AI Supply Chain Security
Assuring the integrity and provenance of everything an AI system is built from and depends on: pre-trained models and weights, adapters, datasets, libraries, serving infrastructure, plug-ins and tools. AI adds components that ordinary software supply-chain controls were not built to check.
Membership Inference
An attack that works out whether a particular person's record was in a model's training data by examining how the model responds. Showing that someone's data was used can itself be a disclosure of personal data.
MITRE ATLAS
MITRE's knowledge base of the tactics and techniques adversaries use against AI systems, modelled on the MITRE ATT&CK framework for conventional cyber attacks and built from real-world case studies and red-team findings.