Inference
Using a trained model to answer questions; a small cost each time, repeated billions of times.
Learn it through a story
- The Tanker and the Data Centre (Jugnupur Tales Vol 2, Ch 9)
Short on time? Read the 5-minute version.
Related terms
- Data centre A building full of computers that run websites, apps and AI.
- GPU A chip that can do huge numbers of calculations at once; AI uses lots of them.
- High-bandwidth memory (HBM) Very fast memory stacked next to AI chips. Making it uses factory capacity that once made phone memory.
- Latency The delay between asking and getting an answer.
- Server A powerful computer, usually with no screen, that does work for many people over the internet.
- Watt-hour (Wh) A measure of energy; 1,000 Wh is one unit on the electricity bill.