How do chatbots like ChatGPT actually work?
Chatbots such as ChatGPT, Gemini and Claude are powered by large language models (LLMs). The core idea is surprisingly simple.
The big idea: predict the next word
Finish this sentence: “Chai with a plate of hot ___”
You probably thought pakoras or samosas. You didn’t look it up. You’ve seen enough sentences to know what usually comes next. An LLM does the same thing, at enormous scale. Given some text, it predicts a likely next word (strictly, a token, a word or piece of a word). Then it adds that word and predicts the next one, and the next, until the answer is done.
Where the “knowledge” comes from
- Pre-training: the model reads a huge amount of text (books, websites, code) and practises predicting the next token trillions of times. Along the way it picks up grammar, facts, reasoning patterns and writing styles.
- Fine-tuning: people then train it further on examples of helpful conversations, and rate its answers so it learns to be useful, polite and safe.
The result is a model that’s very good at producing text that sounds right.
Why that matters
Sounding right and being right aren’t the same thing. Because an LLM generates likely-looking text:
- It can make things up confidently: fake quotes, wrong dates, books that don’t exist. This is called hallucination.
- It has a knowledge cutoff. Unless it’s connected to search, it doesn’t know about recent events.
- It doesn’t “understand” the way you do. It has no experiences, only patterns from text.
Modern chatbots reduce these problems with web search, tools and better training, but they don’t remove them.
Use them well
LLMs are great for explaining ideas, brainstorming, summarising, practising a language and drafting. For facts that matter, ask for sources and check them. Read Why AI makes mistakes next.
Check yourself
At its core, what does a large language model do?
What is a "hallucination" in AI?
Why might a chatbot not know about last week's news?
What is fine-tuning for?
Go deeper
- Video Large language models explained briefly ↗ · 3Blue1Brown
- Video Transformers, the tech behind LLMs ↗ · 3Blue1Brown
More for you: 🎒 Students · 🧑🏫 Parents and teachers · 📰 Journalists and media · ✍️ Bloggers and writers · 🧭 Job seekers