NVIDIA GPUs
NVIDIA GeForce RTX 3060 12 GB
- Memory
- 12 GB
- Bandwidth
- 360 GB/s
Largest popular model at 4-bit: Gemma 4 12B
The cheapest way into CUDA with 12 GB. Slow, but enough for 7–12B models.
from $329 (launch price)Good budget option
NVIDIA · Edge AI · 2024
NVIDIA Jetson Orin Nano Super Developer Kit
Illustration
Good budget option
A 25 W CUDA board that runs 1–4B models: robotics and smart-home projects, not chat at scale.
- Memory
- 8 GB
- Bandwidth
- 102 GB/s
- Power
- 25 W
- Largest popular model at 4-bit
- None of our popular models
from $399 (typical price)
Researched from manufacturer data and public sources; not tested by us.· Last reviewed Oct 11, 2026
What it can run
EstimatePopular models at Q4_K_M with a 32K-token context. Memory needed from real model files; speed is the theoretical ceiling.
Change model size or context
Model weightsContext cache (KV)RuntimeUsable limit (colour = verdict)
| Model | Verdict | Memory | Speed ceiling |
|---|---|---|---|
| Qwen3 1.7B | Not enough memory | — | |
| Qwen3.5 9B | Not enough memory | — | |
| Gemma 4 12B | Not enough memory | — | |
| gpt-oss-20b | Not enough memory | — | |
| Qwen3.6 27B | Not enough memory | — | |
| Qwen3.6 35B A3B | Not enough memory | — | |
| Llama 3.3 70B | Not enough memory | — | |
| gpt-oss-120b | Not enough memory | — | |
| Qwen3.5 122B A10B | Not enough memory | — | |
| Qwen3.5 397B A17B | Not enough memory | — |
Theoretical maximum for generating text: memory bandwidth divided by the bytes read per token. Real runs usually reach about half to three quarters of it.
Where to buy
Prices are editorial references (launch price and a typical street price on the date shown), not live prices. In 2026 the memory shortage has pushed many prices well above launch.
- Jetson Orin Nano Super Developer Kit, 8 GB8 GBLaunch price: $249 · Typical price: ~$399 (checked Jul 22, 2026)
- Check price Amazon.com (opens in a new tab)Amazon.comCheck availability
Other regions (3)
- Check price Amazon.co.uk (opens in a new tab)Amazon.co.ukUnited Kingdom · Check availability · Search results
- Check price Amazon.es (opens in a new tab)Amazon.esSpain · Check availability
- Check price Amazon.de (opens in a new tab)Amazon.deGermany · Check availability · Search results
Prices are shown only if updated in the last 24 hours and may change; the retailer’s price at checkout applies. We may earn a commission from these links at no cost to you. Affiliate disclosure
Overview
Editorial opinionBest for: Voice assistants, vision plus small language models, and learning CUDA at low power.
Editorial opinionWould we buy it today?
For projects and tinkering; for a home chat server a used GPU or a Mac mini is better.
Pros and cons
Editorial opinionPros
- CUDA and Ollama on a 25 W board
- Good for vision + small LLM projects
Cons
- 8 GB shared with the OS
- 102 GB/s: slow beyond 4B models
- Price rose to $399 in 2026
Specifications
Factual data- Manufacturer
- NVIDIA
- Chip
- Orin Nano
- Architecture
- Ampere GPU + Arm Cortex-A78AE
- Type
- Edge board / accelerator
- Form factor
- Developer kit / board
- Released
- 2024
- Availability
- On sale
- Memory options
- 8 GB
- Memory type
- LPDDR5 unified
- Memory bus
- 128-bit
- Memory bandwidth
- 102 GB/s
- Usable for models (default)
- 6 GB / 8 GB
- INT8 AI performance
- 67 TOPS
- Board / max power
- 25 W
- Compute stack
- CUDA (JetPack)
- Models
- Any model that fits
- Several units for one model
- No
Source: Manufacturer specifications
Software support
Editorial opinionHow well the popular local-AI tools run on it today (editorial assessment).
- llama.cppGood
- OllamaGood
- LM StudioNot supported
- vLLMWorks with effort
- MLXNot supported
Measured speeds
Factual dataMeasured results with their source. We never invent speeds.
No measured results yet. The table above shows the theoretical ceiling instead.
Alternatives
Intel GPUs
Intel Arc B580
- Memory
- 12 GB
- Bandwidth
- 456 GB/s
Largest popular model at 4-bit: Gemma 4 12B
12 GB with decent bandwidth for little money; software is the catch.
from $249 (launch price)Good budget option
NVIDIA GPUs
NVIDIA GeForce RTX 5070
- Memory
- 12 GB
- Bandwidth
- 672 GB/s
Largest popular model at 4-bit: Gemma 4 12B
A gaming card with only 12 GB. For AI the 16 GB RTX 5060 Ti is the better buy.
from $1,010 (typical price)Not recommended for AI