Pre-training
The first stage, learning from a huge amount of text by guessing the next word.
Learn it through a story
- How a Chatbot Goes to School (Jugnupur Tales Vol 2, Ch 5)
Short on time? Read the 5-minute version.
Related terms
- Next-word prediction Guessing the word most likely to come next.
- Noise Random dots with no picture in them.
- Open-weight model A model whose trained weights are published, so anyone can download and run it.
- Probability How likely something is to happen.
- Prompt The words you give an AI to tell it what to make.
- Prompt engineering The skill of writing clear instructions so AI gives useful answers.