NVIDIA GPUs
NVIDIA GeForce RTX 3090
- Memory
- 24 GB
- Bandwidth
- 936 GB/s
Largest popular model at 4-bit: Qwen3.6 27B
The homelab classic: 24 GB and 936 GB/s, bought used. Two of them make 48 GB.
from $1,340 (typical price)Good budget option
NVIDIA · NVIDIA GPUs · 2025
NVIDIA GeForce RTX 5090
Photo: PantheraLeo1359531 · CC BY 4.0
Recommended
The fastest consumer card for local AI: 32 GB and 1.8 TB/s make 30B-class models fly.
- Memory
- 32 GB
- Bandwidth
- 1.79 TB/s
- Power
- 575 W
- Largest popular model at 4-bit
- Qwen3.6 35B A3B
from $4,329 (typical price)
Researched from manufacturer data and public sources; not tested by us.· Last reviewed Oct 11, 2026
What it can run
EstimatePopular models at Q4_K_M with a 32K-token context. Memory needed from real model files; speed is the theoretical ceiling.
Change model size or context
Model weightsContext cache (KV)RuntimeUsable limit (colour = verdict)
| Model | Verdict | Memory | Speed ceiling |
|---|---|---|---|
| Qwen3 1.7B | Runs | ≤ 1,614 tok/s | |
| Qwen3.5 9B | Runs | ≤ 315 tok/s | |
| Gemma 4 12B | Runs | ≤ 252 tok/s | |
| gpt-oss-20b | Runs | ≤ 895 tok/s | |
| Qwen3.6 27B | Runs | ≤ 107 tok/s | |
| Qwen3.6 35B A3B | Runs | ≤ 972 tok/s | |
| Llama 3.3 70B | Not enough memory | — | |
| gpt-oss-120b | Partly in system RAM | — | |
| Qwen3.5 122B A10B | Partly in system RAM | — | |
| Qwen3.5 397B A17B | Partly in system RAM | — |
Theoretical maximum for generating text: memory bandwidth divided by the bytes read per token. Real runs usually reach about half to three quarters of it.
Where to buy
Prices are editorial references (launch price and a typical street price on the date shown), not live prices. In 2026 the memory shortage has pushed many prices well above launch.
- RTX 5090 32 GB (any partner card)32 GBLaunch price: $1,999 · Typical price: ~$4,329 (checked Jul 11, 2026)
- Check price Amazon.com (opens in a new tab)Amazon.comCheck availability
Other regions (3)
- Check price Amazon.co.uk (opens in a new tab)Amazon.co.ukUnited Kingdom · Check availability · Search results
- Check price Amazon.es (opens in a new tab)Amazon.esSpain · Check availability
- Check price Amazon.de (opens in a new tab)Amazon.deGermany · Check availability · Search results
Prices are shown only if updated in the last 24 hours and may change; the retailer’s price at checkout applies. We may earn a commission from these links at no cost to you. Affiliate disclosure
Overview
Editorial opinionBest for: Speed with 27–32B models and image or video generation, if the budget allows.
Editorial opinionWould we buy it today?
Only if speed matters more than size. For bigger models, 128 GB unified-memory machines cost less per GB.
Pros and cons
Editorial opinionPros
- Highest memory bandwidth of any consumer card
- CUDA: every tool and runtime works on day one
- 32 GB fits the best 27–31B models with long contexts
Cons
- Street prices in 2026 are about double the $1,999 launch price
- 575 W: needs a big power supply and good airflow
- 32 GB is not enough for 70B+ models on its own
Specifications
Factual data- Manufacturer
- NVIDIA
- Chip
- GB202
- Architecture
- Blackwell
- Type
- Graphics card
- Form factor
- PCIe card
- Released
- 2025
- Availability
- On sale
- Memory options
- 32 GB
- Memory type
- GDDR7
- Memory bus
- 512-bit
- Memory bandwidth
- 1.79 TB/s
- Usable for models (default)
- 32 GB / 32 GB
- Board / max power
- 575 W
- Interface
- PCIe 5.0 x16
- Slots
- 2
- Power connector
- 16-pin 12V-2x6
- Compute stack
- CUDA
- Models
- Any model that fits
- Several units for one model
- Yes
Source: Manufacturer specifications
Software support
Editorial opinionHow well the popular local-AI tools run on it today (editorial assessment).
- llama.cppExcellent
- OllamaExcellent
- LM StudioExcellent
- vLLMExcellent
- MLXNot supported
Measured speeds
Factual dataMeasured results with their source. We never invent speeds.
No measured results yet. The table above shows the theoretical ceiling instead.
Alternatives
Two used cards give 48 GB for less
AMD GPUs
AMD Radeon AI PRO R9700
- Memory
- 32 GB
- Bandwidth
- 645 GB/s
Largest popular model at 4-bit: Qwen3.6 35B A3B
32 GB for about a third of an RTX 5090: the cheapest new 32 GB card.
from $1,550 (typical price)Recommended
Same 32 GB for about a third of the price
Apple silicon
Apple Mac Studio (M5 Max)
- Memory
- 128 GB
- Bandwidth
- 614 GB/s
Largest popular model at 4-bit: Qwen3.5 122B A10B
128 GB at 614 GB/s: gpt-oss-120b and 122B MoE models at a good pace, almost silently.
from $2,499 (launch price)Highly recommended
Four times the memory, quieter, slower