Overview
A large language model (LLM) is a neural network trained on enormous amounts of text to predict the next piece of text — a 'token' — given what came before. That single, simple objective, applied at massive scale, turns out to produce systems that can write, summarise, translate, code, and reason to a surprising degree.
The key surprise of LLMs is emergence: capabilities the designers didn't explicitly build appear once the model and its training data grow large enough. Nobody programmed grammar or arithmetic in; they emerged from learning to predict text well.
How it works
Pre-training
The model reads a huge corpus and learns to predict the next token, absorbing patterns of language, facts, and reasoning.
Fine-tuning
It's refined on higher-quality examples and human feedback to be helpful, follow instructions, and behave safely.
Inference
When you prompt it, the model generates a response one token at a time, each conditioned on everything so far.
Context and tools
Modern LLMs can take in long context, call tools, and act in loops — turning a text predictor into a general assistant.
A concrete example
Ask an LLM to summarise a contract and it doesn't 'look up' an answer — it generates the most probable helpful summary token by token, drawing on patterns learned from millions of documents. That's why it's fluent and useful, and also why it can occasionally state something false with total confidence.
Limits & risks
- Hallucination — producing fluent, confident text that is simply wrong.
- No inherent grounding in truth; the objective is plausibility, not accuracy.
- Sensitivity to how a prompt is phrased, and limits on what fits in context.
Frequently asked questions
Do LLMs 'understand' language?
They model statistical patterns of language extremely well, which produces behaviour that often looks like understanding — but whether that constitutes genuine understanding is debated.
Why do LLMs make things up?
Because they're trained to produce plausible text, not verified facts. Without grounding, a confident-but-wrong answer can be as probable as a correct one.