Hardware → models
RTX 2060 6 GB
Start with Phi-4-mini-instruct at Q4_K_M for general chat. For a coding-focused option, try Qwen2.5-Coder 7B.
Models to start with
Planning estimates6 GB dedicated VRAM · 4,000 tokens · 32 GB system RAM · full GPU residency
| Model / starting file | Why consider it | GPU memory estimate | Memory left¹ |
|---|---|---|---|
| Phi-4-mini-instructQ4_K_M | Small instruction-following text modelUses a different checkpoint and chat template from Phi reasoning or vision variants. | 4.9 GB | 0.9 GB |
| Qwen2.5-Coder 7BQ4_K_M | Coding with a smaller memory budgetA coding-focused model. Run and review generated code before using it. | 5.8 GB | 0.0 GB |
| Llama 3.2 3B InstructQ4_K_M | Small text assistantA smaller weight footprint. Check the exact Llama checkpoint and licence. | 3.7 GB | 2.1 GB |
From the published model guides here; the Finder covers the full catalogue. ¹ Memory left after the Finder’s reserve. Other applications can use this space. Larger models and higher precision do not guarantee better answers.
This page uses the 6 GB card. Keep context modest; the 7B Coder fits tightly in this estimate, while the listed 14B Q4 profiles exceed the budget. CPU offloading changes the memory allocation and may slow generation.
Setup and common questions
Which RTX 2060 is this?
This page covers the original 6 GB RTX 2060. Other 2060 variants have different memory. Match the exact VRAM shown by your hardware; a shared family name is not enough.
What should I change if memory is tight?
Start at 4K context, close other GPU-heavy applications and check the runtime’s reported allocation. Try a smaller model before adding a larger cache. Display use and runtime buffers can take more memory than the planning reserve.
Can CPU offloading make a larger model usable?
It can move some weights into system RAM when the runtime supports it. That is a different setup from this full-GPU table and can slow generation. Use the Finder to compare smaller models, then test your exact configuration rather than treating a successful load as useful speed.
How are these recommendations chosen?
We use the main Finder’s balanced task selection and memory calculation, limited to models with published guides here. Task scores and speed proxies are planning inputs, not measured quality rankings.
GPU memory includes weights, context cache and a 0.9 GB runtime allowance, plus a 3% reserve (0.15–1.5 GB). Cache format and runtime buffers can change the fit. See the calculation assumptions.
Sources and related guides
Specification sources checked 3 October 2026.
These comparisons are planning estimates. No reviewed or independently reproduced measurements are available yet. About test evidence.