Hardware to run Llama 3.1 8B
The classic 8B model. Still one of the most downloaded; newer 8–9B models are stronger.
- Developer
- Meta
- Parameters
- 8B
- Max context
- 128K tokens
- Licence
- Llama 3.1 Community License
Model card on Hugging FaceGGUF filesOllama: ollama run llama3.1:8b
Memory needed
Factual dataWeights plus context cache plus about 1.5 GB of runtime buffers, by quantisation and context length.
| Quantisation | 8K tokens | 32K tokens | 128K tokens |
|---|---|---|---|
| Q4_K_M | 7.5 GB | 11 GB | 24 GB |
| Q8_0 | 11 GB | 14 GB | 27 GB |
| FP16 | 19 GB | 22 GB | 35 GB |
Hardware that runs it
Estimate31 platforms can run Llama 3.1 8B at Q4_K_M with a 32K-token context. Needs about 11 GB.
Cheapest that runs it
Fastest that runs it
Model weightsContext cache (KV)RuntimeUsable limit (colour = verdict)
RunsRuns comfortably (30)
| Platform | Smallest configuration | Memory | Speed ceiling | Price from |
|---|---|---|---|---|
| RTX PRO 6000 | 96 GB | ≤ 364 tok/s | $8,565 | |
| RTX 5090 | 32 GB | ≤ 364 tok/s | $4,329 | |
| RTX PRO 5000 | 48 GB | ≤ 273 tok/s | $4,500 | |
| Mac Studio M5 Ultra | 96 GB | ≤ 244 tok/s | $5,499 | |
| RTX 4090 | 24 GB | ≤ 205 tok/s | $1,599 | |
| RX 7900 XTX | 24 GB | ≤ 195 tok/s | $1,358 | |
| RTX 5080 | 16 GB | ≤ 195 tok/s | $1,830 | |
| RTX 3090 | 24 GB | ≤ 190 tok/s | $1,340 | |
| RTX PRO 4500 | 32 GB | ≤ 182 tok/s | $2,500 | |
| RTX 5070 Ti | 16 GB | ≤ 182 tok/s | $1,190 | |
| RTX PRO 4000 | 24 GB | ≤ 137 tok/s | $1,500 | |
| RTX 5070 | 12 GB | ≤ 137 tok/s | $1,010 | |
| Radeon AI PRO R9700 | 32 GB | ≤ 131 tok/s | $1,550 | |
| RX 9070 XT | 16 GB | ≤ 131 tok/s | $759 | |
| Arc Pro B70 | 32 GB | ≤ 124 tok/s | $1,268 | |
| Mac Studio M5 Max | 36 GB | ≤ 93 tok/s | $2,499 | |
| Arc Pro B60 | 24 GB | ≤ 93 tok/s | $599 | |
| Arc B580 | 12 GB | ≤ 93 tok/s | $249 | |
| RTX 5060 Ti 16 GB | 16 GB | ≤ 91 tok/s | $789 | |
| RTX 3060 12 GB | 12 GB | ≤ 73 tok/s | $329 | |
| RX 9060 XT 16 GB | 16 GB | ≤ 65 tok/s | $349 | |
| Mac mini M5 Pro | 24 GB | ≤ 62 tok/s | $1,699 | |
| Ryzen AI Max+ PRO 495 | 192 GB | ≤ 55 tok/s | $7,399 | |
| Jetson AGX Thor | 128 GB | ≤ 55 tok/s | $5,499 | |
| DGX Spark / GB10 | 128 GB | ≤ 55 tok/s | $5,000 | |
| Ryzen AI Max+ 395 | 64 GB | ≤ 52 tok/s | $2,200 | |
| Arc Pro B50 | 16 GB | ≤ 46 tok/s | $349 | |
| Jetson AGX Orin 64GB | 64 GB | ≤ 42 tok/s | $1,999 | |
| Ryzen AI 9 HX 370 | 32 GB | ≤ 24 tok/s | — | |
| RK3588 (Orange Pi 5 Plus and others) | 16 GB | — | — |
Runs, tightRuns, but tight (1)
| Platform | Smallest configuration | Memory | Speed ceiling | Price from |
|---|---|---|---|---|
| Mac mini M6 | 16 GB | ≤ 35 tok/s | $899 |
Not enough memoryNot enough memory (1)
| Platform | Smallest configuration | Memory | Speed ceiling | Price from |
|---|---|---|---|---|
| Jetson Orin Nano Super | 8 GB | — | $399 |
Not supportedNot supported (1)
| Platform | Smallest configuration | Memory | Speed ceiling | Price from |
|---|---|---|---|---|
| Raspberry Pi AI HAT+ 2 | 8 GB | — | — | $200 |
Theoretical maximum for generating text: memory bandwidth divided by the bytes read per token. Real runs usually reach about half to three quarters of it.