For Heads-of · Practitioner
Context Window
The maximum length of text (measured in tokens) that a language model can consider at once when generating a response.
- context window
- token limit
- model capacity
Limitation
A model cannot see anything beyond its context window. Information outside it, earlier in a long conversation or further into a long document, is simply not available to the model when it answers.
Trade-offs
Larger windows let a model work with longer documents and conversations, but cost more and respond more slowly, and models often attend less reliably to material in the middle of a very long context. Long contexts also widen the attack surface for injected instructions and many-shot jailbreaks.
Governance impact
What a model can consider bounds what its output can be trusted to reflect: a system cannot apply a policy document it never received in full. Long contexts also drive cost and can be abused to run it up, the unbounded resource consumption risk that inference usage quotas and rate limiting address.
Related Terms
Large Language Model (LLM)
A deep learning model trained on very large amounts of text, able to generate and interpret human-like language across a wide range of topics and tasks.
Prompt Injection
An attack in which untrusted content reaching a language model is interpreted as instructions rather than data, causing the model to act on an attacker's intent instead of the operator's. It is a property of how the application assembles its context, not a flaw in any particular model.
Retrieval-Augmented Generation (RAG)
A technique in which a language model retrieves relevant documents or data before generating a response, grounding its output in source material rather than in its training data alone.
Token
A unit of text that a language model processes, typically a word, part of a word or a character. Models work on sequences of tokens, not raw text.