Guides
Answers to the questions people ask most about running LLMs locally, with the measurements behind them.
Which local LLM should I run on a 16 GB GPU?
Short answer: a ~27B dense model at about 3.5 bits, or a 26B MoE at 4 bits. Measured on an RX 9070 XT, with scores per language.
Why is llama.cpp slow on my AMD or Intel GPU on Windows?
Usually Resizable BAR. Without it, llama.cpp's Vulkan backend can put buffers in a small host-visible heap, and decode gets up to 1.7x slower. Here is what to check and the fix.
The best local LLM for Thai, Hindi, Arabic and other languages
Measured scores in seven languages, why quantization hurts some languages more, and how to test any model in your own language in a few minutes.
How much RAM does a local LLM need?
With the model on the GPU, about 2.2 GB of system RAM, if the caches are sized for one user. With llama-server's defaults it grows to about 9 GB in a long chat.
Local AI or cloud AI: when to use which
A local model on your GPU is private, free per token and works offline; a frontier cloud model knows more and reasons better. Most people need both. Here is how to split the work, with our measurements.
Run a coding agent on a local model (Hermes, OpenCode, Aider, Claude Code)
Three commands point an agent at the model on your GPU, with a context large enough for agent prompts. What works, what to expect, and how to go back.
Use Obsidian with a local LLM, without sending notes to the cloud
Copilot for Obsidian can talk to the model on your GPU. One command sets it up and tells you which settings would still send notes to a cloud service.