Local LLM Finder

Local LLM Finder Knowledge

What can you run?

Start with your hardware or a model you want to try.

Choose your hardware

Different setup? Use the Finder
RTX 3060 12 GB12 GB VRAM · model starting pointsRTX 4060 Ti 16 GB16 GB VRAM · model starting pointsRTX 4090 24 GB24 GB VRAM · model starting pointsRTX 5090 32 GB32 GB VRAM · model starting pointsRTX 6000 Ada 48 GB48 GB VRAM · model starting pointsRX 7900 XTX 24 GB24 GB VRAM · model starting pointsIntel Arc B580 12 GB12 GB VRAM · model starting pointsRTX 2060 6 GB6 GB VRAM · model starting pointsRTX 3080 10 GB10 GB VRAM · model starting pointsRTX 4060 8 GB8 GB VRAM · model starting points

Choose a model

Qwen3 8BGeneral chat and mixed tasksQwen2.5-Coder 14BCoding with a larger modelgpt-oss-20bReasoning and tool-oriented tasksLlama 3.2 3B InstructSmall text assistantQwen3 4BSmall general-purpose modelQwen3 14BGeneral chat with a larger modelQwen3 32BLarger general-purpose modelQwen2.5-Coder 7BCoding with a smaller memory budgetQwen2.5-Coder 32BLarger dense coding modelGemma 3 12BText inference with Gemma 3Phi-4-mini-instructSmall instruction-following text modelQwen3-Coder 30B-A3B InstructCoding mixture-of-experts model

Understand the settings

Q4 or Q5?Choose a quantisationHow much context?Plan space for prompts and replies

These pages offer planning estimates. For your exact setup and task, use the main Finder.