For Heads-of · Practitioner
Transformer
A deep learning architecture that uses self-attention to process sequences of data, and the foundation of modern large language models.
- transformers
- attention
- architecture
- LLM
How it works
A transformer processes all the tokens in its input together rather than one at a time, using attention to weigh which parts of the input matter most for each part of the output.
Efficiency
Processing in parallel made transformers faster to train and more scalable than earlier architectures such as recurrent neural networks (RNNs), which is what made today's very large models practical.
Dominance in LLMs
The major large language models are built on transformer architectures. One consequence matters for security: instructions and data arrive in the same token sequence and pass through the same attention mechanism, with no separate channel for either. That is the architectural root of prompt injection.
Related Terms
Embedding
A numerical representation of text, images or other data in a high-dimensional space, where items with similar meaning sit close together.
Large Language Model (LLM)
A deep learning model trained on very large amounts of text, able to generate and interpret human-like language across a wide range of topics and tasks.
Neural Network
A computational model loosely inspired by biological neurons, made of interconnected layers of artificial nodes that learn to recognise patterns in data.
Token
A unit of text that a language model processes, typically a word, part of a word or a character. Models work on sequences of tokens, not raw text.