For Heads-of · Practitioner
Inference usage quotas and rate limiting
Per-user and per-tenant limits on request volume, input length, and context-window usage for an AI system's inference endpoint.
- preventive
- availability
- cost-control
- rate-limiting
What it does
Enforces quotas and rate limits on inference requests, input size, and context-window usage, with alerting when a caller approaches the limit.
Where it fits
The direct mitigation for unbounded-resource-consumption (OWASP LLM10), and a meaningful speed bump against model-extraction querying patterns.
Risks this mitigates
The risks this control addresses, ranked by effectiveness.
Model extraction and inference-API abuse
Repeated querying of a model's inference API extracts private training data or effectively replicates the model itself.
Unbounded resource consumption
Unrestricted or excessive inference requests lead to denial of service, denial of wallet, or facilitate model theft.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| OWASP Top 10 for LLM Applications | LLM06 | Unbounded Consumption |