Latency
The delay between asking and getting an answer.
For example
On-device AI can reply faster than a cloud model because nothing travels over the internet.
Related terms
- Data centre A building full of computers that run websites, apps and AI.
- GPU A chip that can do huge numbers of calculations at once; AI uses lots of them.
- High-bandwidth memory (HBM) Very fast memory stacked next to AI chips. Making it uses factory capacity that once made phone memory.
- Inference Using a trained model to answer questions; a small cost each time, repeated billions of times.
- Server A powerful computer, usually with no screen, that does work for many people over the internet.
- Watt-hour (Wh) A measure of energy; 1,000 Wh is one unit on the electricity bill.