Some links are affiliate links: if you buy through them we may earn a commission, at no extra cost to you. It never changes our assessments. Affiliate disclosure
Two numbers decide almost everything in local AI: memory capacity (which models fit) and memory bandwidth (how fast they answer). Graphics cards and unified-memory computers sit at opposite ends of that trade-off.
Graphics cards: fast but small
A graphics card has its own very fast memory: about 0.9 TB/s on an NVIDIA GeForce RTX 3090, 1.8 TB/s on an RTX 5090. But consumer cards stop at 24–32 GB, and in 2026 they are expensive. Two or more cards add capacity, at the cost of power, heat and a bigger case.
Unified memory: big but slower
In unified-memory machines the CPU and GPU share one large pool of memory. That makes 128 GB or more affordable in a small box:
| Machine | Memory | Bandwidth |
|---|---|---|
| AMD Ryzen AI Max+ 395 (Strix Halo) | up to 128 GB | 256 GB/s |
| NVIDIA GB10 Grace Blackwell (DGX Spark and partners) | 128 GB | 273 GB/s |
| Apple Mac Studio (M5 Max) | up to 128 GB | 614 GB/s |
| Apple Mac Studio (M5 Ultra) | up to 512 GB | 1.2 TB/s |
The bandwidth is lower than a high-end GPU's, except on the Ultra.
What that means in practice
Speed ceilings at 4-bit (bandwidth ÷ bytes read per token; real runs reach roughly half to three quarters):
| Qwen3.6 27B (dense) | gpt-oss-120b (MoE) | |
|---|---|---|
| RTX 3090 (24 GB) | ≤ 56 tok/s | does not fit |
| Ryzen AI Max+ 395 (128 GB) | ≤ 15 tok/s | ≤ 93 tok/s |
| DGX Spark / GB10 (128 GB) | ≤ 16 tok/s | ≤ 100 tok/s |
| Mac Studio M5 Max (128 GB) | ≤ 37 tok/s | ≤ 224 tok/s |
Two lessons:
- Dense models favour GPUs. Every token reads all 17 GB of Qwen3.6 27B, so bandwidth rules. A used RTX 3090 beats every unified-memory box except the Mac Studio Ultra here.
- Mixture-of-experts models favour unified memory. gpt-oss-120b reads only its 5.1B active parameters per token, so even 256 GB/s is fast, but you need about 65 GB to hold it. That is exactly what these machines have and single GPUs lack.
The best open models of 2026 are increasingly MoE (Qwen3.6 35B A3B, gpt-oss, Qwen3.5 122B), which is why 128 GB machines have become the homelab favourite.
Prompt processing
Reading a long prompt (a document, a codebase) depends on compute rather than bandwidth. Here NVIDIA GPUs and the GB10 are well ahead of Strix Halo and Macs. If you paste long documents often, weigh that in.
How to choose
- Mostly 8–32B models, want speed: a 24–32 GB graphics card.
- Big MoE models, quiet 24/7 box, best value: AMD Ryzen AI Max+ 395 (Strix Halo) with 128 GB.
- Same, but faster and with no tinkering: Apple Mac Studio (M5 Max) with 128 GB.
- You need CUDA (vLLM, fine-tuning): NVIDIA GB10 Grace Blackwell (DGX Spark and partners).