Hardware → models
RTX 3080 10 GB
Start with Qwen3 8B at Q4_K_M for general chat. For a coding-focused option, try Qwen2.5-Coder 7B.
Models to start with
Planning estimates10 GB dedicated VRAM · 4,000 tokens · 32 GB system RAM · full GPU residency
| Model / starting file | Why consider it | GPU memory estimate | Memory left¹ |
|---|---|---|---|
| Qwen3 8BQ4_K_M | General chat and mixed tasksA middle-sized option with more memory headroom than the 14B models. | 6.2 GB | 3.5 GB |
| Qwen2.5-Coder 7BQ4_K_M | Coding with a smaller memory budgetA coding-focused model. Run and review generated code before using it. | 5.8 GB | 3.9 GB |
| Llama 3.2 3B InstructQ4_K_M | Small text assistantA smaller weight footprint. Check the exact Llama checkpoint and licence. | 3.7 GB | 6.0 GB |
From the published model guides here; the Finder covers the full catalogue. ¹ Memory left after the Finder’s reserve. Other applications can use this space. Larger models and higher precision do not guarantee better answers.
This page uses the 10 GB variant. The listed 14B Q4 profiles exceed this full-GPU budget; the 12 GB variant has a different limit. CPU offloading changes the memory allocation and may slow generation.
Setup and common questions
Why does the exact RTX 3080 variant matter?
NVIDIA lists 10 GB and 12 GB RTX 3080 variants. This page uses 10 GB. A recommendation calculated for the 12 GB version can exceed this card’s budget; select the exact model in the Finder.
Should I try a 14B model?
At the Q4 and 4K settings shown, the reviewed 14B profiles exceed this full-GPU budget. Start with the listed smaller models. More GPU compute does not make weights and cache fit into less VRAM.
Can I increase Q4 to Q5?
Check the exact model’s Q5 profile in the Finder first, keeping context fixed. It uses more weight memory and reduces headroom. Compare answers on your own task; higher precision does not guarantee a useful improvement, and this page does not establish inference speed.
How are these recommendations chosen?
We use the main Finder’s balanced task selection and memory calculation, limited to models with published guides here. Task scores and speed proxies are planning inputs, not measured quality rankings.
GPU memory includes weights, context cache and a 0.9 GB runtime allowance, plus a 3% reserve (0.15–1.5 GB). Cache format and runtime buffers can change the fit. See the calculation assumptions.
Sources and related guides
Specification sources checked 3 October 2026.
These comparisons are planning estimates. No reviewed or independently reproduced measurements are available yet. About test evidence.