Model → hardware
Llama 3.2 3B Instruct
Plan for 4 GB of dedicated GPU memory at Q4_K_M and a 4,000-token context. 6 GB gives more headroom under the same settings.
Small text assistant. Memory fit is an estimate, not a speed or quality guarantee.
Hardware starting points
Planning estimatesQ4_K_M · 32 GB system RAM · one active request · full GPU residency
| Dedicated GPU memory | At 4,000 tokens | At 16,000 tokens | Example from these guides |
|---|---|---|---|
| 4 GB | Smallest listed tier that fits3.7 GB + 0.15 GB reserve | Exceeds this budget4.1 GB + reserve | Compare your exact hardware in the Finder |
| 6 GB | More headroom3.7 GB + 0.18 GB reserve | Fits the estimate4.1 GB + reserve | RTX 2060 6 GB |
| 8 GB | More headroom3.7 GB + 0.24 GB reserve | Fits the estimate4.1 GB + reserve | RTX 4060 8 GB |
The smallest tier fits the calculation; it is not a verified minimum. Cache precision and runtime buffers can change the fit. “More headroom” means at least 2 GB remains after the estimate and reserve.
The download is a GGUF conversion; Q5 in the table is an estimate, not a listed file in that repository. Use the publisher’s Instruct / chat template in a current GGUF-capable runtime. The 4 GB tier can become tight when display use, buffers or context grow. CPU offloading needs adequate system RAM and changes performance; it is not included in these full-GPU estimates.
Compare quantisation and context
| File profile | Weight input¹ | GPU estimate at 4K | GPU estimate at 16K |
|---|---|---|---|
| Q4_K_M | 2.70 GB | 3.7 GB | 4.1 GB |
| Q5_K_M | 3.30 GB | 4.3 GB | 4.7 GB |
¹ Catalogue planning weights. These can differ from an exact downloaded file; runtime memory also includes cache and buffers.
Keep the exact file, runtime version, cache type and context limit consistent when comparing. Q5 uses more memory; compare its answers on your own tasks before accepting the trade-off.
Setup and common questions
Which Llama checkpoint is this?
Use Llama-3.2-3B-Instruct, the text-only assistant model. LM Studio Community provides a matching Q4_K_M GGUF conversion. Its file is about 2.02 GB; the Finder uses a conservative 2.70 GB weight input before cache and buffers. Check the chat template.
Why do different downloads list different context limits?
Meta lists 128K for the original text checkpoint and 8K for its separate quantised editions. A converted GGUF is not automatically that official quantised edition. Start at 4K; the listed maximum does not establish a memory fit.
What licence applies?
The Llama 3.2 Community License and Acceptable Use Policy apply. Review Meta’s terms before distributing a model or building a service. This page’s Q4 weight input is a planning estimate, not a verified download size.
How are these recommendations chosen?
We compare dedicated GPU memory tiers using the main Finder’s memory calculation at the stated quantisation and context. These tiers describe memory fit, not speed or output quality.
GPU memory includes weights, context cache and a 0.9 GB runtime allowance, plus a 3% reserve (0.15–1.5 GB). Cache format and runtime buffers can change the fit. See the calculation assumptions.
Sources and related guides
Specification sources checked 3 October 2026.
These comparisons are planning estimates. No reviewed or independently reproduced measurements are available yet. About test evidence.