AMD GPUs
AMD Radeon AI PRO R9700
- Memory
- 32 GB
- Bandwidth
- 645 GB/s
Largest popular model at 4-bit: Qwen3.6 35B A3B
32 GB for about a third of an RTX 5090: the cheapest new 32 GB card.
from $1,550 (typical price)Recommended
NVIDIA · NVIDIA GPUs · 2022
NVIDIA GeForce RTX 4090
Illustration
For advanced usersDiscontinued (still sold)
Out of production but still a top AI card: 24 GB at 1 TB/s.
- Memory
- 24 GB
- Bandwidth
- 1.01 TB/s
- Power
- 450 W
- Largest popular model at 4-bit
- Qwen3.6 27B
from $1,599 (launch price)
Researched from manufacturer data and public sources; not tested by us.· Last reviewed Oct 11, 2026
What it can run
EstimatePopular models at Q4_K_M with a 32K-token context. Memory needed from real model files; speed is the theoretical ceiling.
Change model size or context
Model weightsContext cache (KV)RuntimeUsable limit (colour = verdict)
| Model | Verdict | Memory | Speed ceiling |
|---|---|---|---|
| Qwen3 1.7B | Runs | ≤ 908 tok/s | |
| Qwen3.5 9B | Runs | ≤ 177 tok/s | |
| Gemma 4 12B | Runs | ≤ 142 tok/s | |
| gpt-oss-20b | Runs | ≤ 504 tok/s | |
| Qwen3.6 27B | Runs | ≤ 60 tok/s | |
| Qwen3.6 35B A3B | Partly in system RAM | — | |
| Llama 3.3 70B | Not enough memory | — | |
| gpt-oss-120b | Partly in system RAM | — | |
| Qwen3.5 122B A10B | Partly in system RAM | — | |
| Qwen3.5 397B A17B | Partly in system RAM | — |
Theoretical maximum for generating text: memory bandwidth divided by the bytes read per token. Real runs usually reach about half to three quarters of it.
Where to buy
Prices are editorial references (launch price and a typical street price on the date shown), not live prices. In 2026 the memory shortage has pushed many prices well above launch.
- RTX 4090 24 GB (new old stock or renewed)24 GB · RenewedLaunch price: $1,599
Out of production: check seller ratings and warranty.
- Check price Amazon.com (opens in a new tab)Amazon.comCheck availability
Other regions (3)
- Check price Amazon.co.uk (opens in a new tab)Amazon.co.ukUnited Kingdom · Check availability · Search results
- Check price Amazon.es (opens in a new tab)Amazon.esSpain · Check availability · Search results
- Check price Amazon.de (opens in a new tab)Amazon.deGermany · Check availability · Search results
Prices are shown only if updated in the last 24 hours and may change; the retailer’s price at checkout applies. We may earn a commission from these links at no cost to you. Affiliate disclosure
Overview
Editorial opinionBest for: Buyers who find one at a fair price with warranty.
Editorial opinionWould we buy it today?
Only below the price of a used RTX 3090 pair or an RTX 5090; otherwise skip.
Pros and cons
Editorial opinionPros
- About 1 TB/s: fast generation
- 24 GB fits 27–32B models at 4-bit
Cons
- Discontinued: stock is old or renewed
- 450 W and a large cooler
- Often priced close to an RTX 5090
Specifications
Factual data- Manufacturer
- NVIDIA
- Chip
- AD102
- Architecture
- Ada Lovelace
- Type
- Graphics card
- Form factor
- PCIe card
- Released
- 2022
- Availability
- Discontinued (still sold)
- Memory options
- 24 GB
- Memory type
- GDDR6X
- Memory bus
- 384-bit
- Memory bandwidth
- 1.01 TB/s
- Usable for models (default)
- 24 GB / 24 GB
- Board / max power
- 450 W
- Interface
- PCIe 4.0 x16
- Power connector
- 16-pin 12VHPWR
- Compute stack
- CUDA
- Models
- Any model that fits
- Several units for one model
- Yes
Source: Manufacturer specifications
Software support
Editorial opinionHow well the popular local-AI tools run on it today (editorial assessment).
- llama.cppExcellent
- OllamaExcellent
- LM StudioExcellent
- vLLMExcellent
- MLXNot supported
Measured speeds
Factual dataMeasured results with their source. We never invent speeds.
No measured results yet. The table above shows the theoretical ceiling instead.
Alternatives
AMD GPUs
AMD Radeon RX 7900 XTX
- Memory
- 24 GB
- Bandwidth
- 960 GB/s
Largest popular model at 4-bit: Qwen3.6 27B
24 GB and 960 GB/s: RTX 3090-class memory, bought new.
from $1,358 (typical price)Recommended
Intel GPUs
Intel Arc Pro B70
- Memory
- 32 GB
- Bandwidth
- 608 GB/s
Largest popular model at 4-bit: Qwen3.6 35B A3B
Intel's big Battlemage: 32 GB and 608 GB/s for less than AMD's R9700.
from $1,268 (typical price)Recommended