Three kinds of brains
AI runs on the same three chips found in most computers: the CPU, the GPU, and a new arrival called the NPU. Tap each one to meet it — no jargon, just what it's actually like.
The GeneralistCPU
~5–65 watts- Picture it like this
- Think of an office worker who can do any task — spreadsheets, emails, planning — but handles them one at a time. Fast at switching, brilliant at logic, not built for thousands of repetitive jobs at once.
- What it does for AI
- Runs the everyday logic of your device and coordinates the other chips. For light AI work — a keyboard autocomplete, a quick summarizer — the CPU quietly gets it done.
Where you'll meet it: In every computer, phone, tablet, and console you've ever used.
From pocket to cloud
The same story plays out at every scale. Scroll through the cards — the physics doesn't change, only the size of the audience.
In your pocket
Your phone's AI brain runs on about 5–10 watts — less than a light bulb. To pull that off, it uses a specialist chip (an NPU) that does AI math far more efficiently than the regular processor.
Up in the cloud
The servers behind services like ChatGPT each pull 1,000+ watts. They cram in as much ultra-fast memory as physics allows, inside liquid-cooled racks crunching trillions of operations a second.
The wall everyone hits
Phone or cloud, both face the same problem: AI needs data faster than memory can supply it. An AI is only as fast as its memory — no exceptions.
AI in your phone
Your phone is a supercomputer now. These three chips lead the pack — pick one to see what makes it special.
Snapdragon 8 Elite
Purpose-built to run large language models on your phone. It reads AI text at 70+ tokens per second on-device — roughly the pace of a fast audiobook narrator — without needing a cloud connection.
- Ships with its own Hexagon NPU ('HTP Gen 5').
- Removed the traditional cache hierarchy to feed the NPU faster.
- Dual-channel LPDDR5X memory running at up to 10.7 Gbps.
Full spec sheet — Snapdragon 8 Elite
| Metric | Snapdragon 8 Elite |
|---|---|
| Process Node | TSMC 3nm (N3E) |
| Instruction Set | Armv8 Architecture |
| CPU Configuration | 8 Cores: 2x Oryon @ 4.32 GHz + 6x Oryon @ 3.53 GHz |
| GPU Architecture | Adreno 830 |
| NPU Architecture | Hexagon (HTP Gen 5) |
| Claimed NPU Metrics | >70 tokens/sec (LLMs) |
| Peak Memory Bandwidth | Dual-channel LPDDR5X (~85.3 GB/s aggregate) |
Benchmark scores
| Benchmark | Snapdragon 8 Elite |
|---|---|
| Geekbench 6 Single-Core | ~3,200 |
| Geekbench 6 Multi-Core | ~10,500 |
| AnTuTu 10 Total Score | 3,014,075 – 3,025,991 |
| AnTuTu 10 GPU Score | 1,134,684 |
The cloud giants
To run the biggest models, you need computers the size of a fridge with memory that costs more than a car. Meet the three accelerators doing the heavy lifting right now.
NVIDIA B200
NVIDIA · Blackwell
Two massive chiplets fused into one 1,000-watt package. It's the workhorse behind most frontier AI training, with 9,000 trillion FP4 operations per second and 8 TB/s of memory bandwidth.
- 208 billion transistors across ~1,628 mm² of silicon.
- First-gen FP4 'Transformer Engine' for micro-scaling formats.
AMD MI325X
AMD · Instinct
The story here is memory: a staggering 256 GB of HBM3e. It can hold enormous models in one place, avoiding the slow dance of shuffling weights around. That's why AMD targets cloud inference giants.
- 256 GB of HBM3e — the largest in its class.
- 6 TB/s of peak memory bandwidth.
Google Trillium
Google · TPU v6
Not a GPU at all — an ASIC built for exactly one job: tensor math. Instead of flexible programming, it uses dedicated Matrix Multiply Units coordinated by vector and scalar cores, with optical switches between racks.
- Dedicated Matrix Multiply Units (MXUs) — no general-purpose flexibility.
- Optical circuit switching between chips for low, stable latency.
Full comparison — NVIDIA B200 vs AMD MI325X vs Google Trillium
| Metric | NVIDIA B200 | AMD MI325X | Google Trillium |
|---|---|---|---|
| Architecture | Blackwell (GB100) | CDNA 3 | Custom TPU ASIC Architecture |
| Form Factor | SXM6 / PCIe Module | OAM (Open Accelerator Module) | Custom Megawatt Rack Pod |
| Process Node | TSMC 4N / 5nm | 5nm FinFET | Advanced Custom Process Node |
| Transistor Count / Die Area | 208 Billion / 1,628 mm² aggregate | Multi-Chiplet Interposer Layout | Custom Matrix Tile Layout |
| VRAM Capacity | 180 GB to 192 GB HBM3e | 256 GB HBM3e | ~32 GB per core (~128 GB aggregate) |
| Memory Bandwidth | 7.7 TB/s to 8.19 TB/s | 6.0 TB/s | ~1.6 TB/s per core (~6.4 TB/s aggregate) |
| Thermal Power (TDP) | 1,000 Watts (1.0 kW) | 1,000 Watts (1.0 kW) | Dynamic Pod-Level Power Optimization |
| FP64 Performance | 37.0 TFLOPS | 81.7 TFLOPS | Domain Specific / Matrix Reduced |
The memory wall
Here's the one idea that explains every hardware fight you read about: to write each word, the chip has to re-read essentially the whole model from memory. So memory speed sets your ceiling.
Try the memory wall for yourself
Drag the sliders. The chip can only read about 90 GB/s from memory, and the model needs 8 GB re-read for every single word. That leaves room for roughly…
A patient voice — usable, but you'll feel the pause.
90 ÷ 8 ≈ 11
A simplified rule of thumb — real speed also depends on the model, batching, and memory layout. But the core idea is exactly this: words per second ≈ memory speed ÷ model size.
Why this matters: moving data around costs about 100× more energy than doing the math on it. That's why chips cram memory right next to the compute — and why “memory bandwidth” is the spec everyone brags about.
Smaller numbers, faster AI
A 100-billion-parameter model is a heavy suitcase. The industry's clever trick: store the same brain with smaller numbers, so it fits in faster memory — and the wall gets easier to climb.
From FP32 to FP4, the same model shrinks from 200 GB to 25 GB — four times less baggage, up to eight times faster math. The trade-off: numbers get blurrier, so hardware needs clever tricks to keep the AI smart.
Where it's all heading
Every fix for the memory wall is a bet on one of these four futures.
Want the full story?
Take the guided curriculum — from your first prompt to running models on this very hardware setup.