On an RTX 3060 12GB, llama.cpp gives you the fastest raw tokens per second and the most control, Ollama is the fastest to set up and gives near-parity performance, and LM Studio is the only one with a first-class GUI — at the cost of ~10% throughput. Pick llama.cpp for scripting, Ollama for daily driver, LM Studio for exploring models.
Why compare these three in 2026
Everyone running local LLMs in 2026 has ended up on one of these three tools. They are not the only options — there is vLLM, Text Generation WebUI, koboldcpp, MLC-LLM, and a dozen others — but for the RTX 3060 12GB owner (still the sweet-spot budget card three years after launch), llama.cpp, Ollama, and LM Studio cover 95% of real usage. Each has a different theory of what "usable local inference" means.
- llama.cpp is the reference GGUF runtime. Command-line, tiny, does one thing well.
- Ollama is a friendly wrapper on top of llama.cpp with a REST API, a model registry, and a
docker pull-like UX. - LM Studio is a desktop GUI with a model browser, chat interface, and OpenAI-compatible local server.
All three run the same underlying inference engine at their core (Ollama and LM Studio both wrap llama.cpp), but they diverge on defaults, tokenization, KV cache management, and how much of the GPU they actually use. Those differences add up to real tokens-per-second gaps on a fixed RTX 3060 12GB build.
Test rig
Everything below was measured on the same box:
- CPU: AMD Ryzen 7 5800X at stock clocks
- RAM: 64 GB DDR4-3600
- GPU: MSI GeForce RTX 3060 Ventus 2X 12G
- Storage: Crucial BX500 1TB SATA SSD for models
- OS: Ubuntu 24.04, NVIDIA driver 550.x, CUDA 12.4
Prompts fixed, temperature 0, top_p 1, single stream, warm KV cache, steady-state tokens/sec measured over 512-token generation windows. Models pulled from Hugging Face with matching GGUF quantizations across all three tools.
Key takeaways
- llama.cpp is 8–12% faster than Ollama on identical models and quants, but you build the invocation string.
- Ollama trades 8–12% of throughput for a REST API, model registry, and a five-second install.
- LM Studio is 10–15% slower than llama.cpp on the same quant, but adds a real GUI, a searchable model browser, and a friendly first-time-user path.
- All three run 13–14B models comfortably on a 3060 12GB at Q4_K_M.
- For an OpenAI-compatible endpoint that "just works," LM Studio and Ollama are tied; llama.cpp needs a wrapper.
Benchmark table: tokens per second on the RTX 3060 12GB
Single stream, Q4_K_M weights, 8k context, chat-length prompts (~200 tokens in), 512 tokens out.
| Model | llama.cpp | Ollama | LM Studio |
|---|---|---|---|
| Llama 3.2 3B | ~74 tok/s | ~68 tok/s | ~65 tok/s |
| Mistral 7B | ~48 tok/s | ~43 tok/s | ~41 tok/s |
| Llama 3.1 8B | ~42 tok/s | ~38 tok/s | ~36 tok/s |
| Qwen 2.5 Coder 14B | ~26 tok/s | ~23 tok/s | ~22 tok/s |
| Qwen 2.5 14B | ~24 tok/s | ~21 tok/s | ~20 tok/s |
The gap is consistent: llama.cpp on top, Ollama within 10%, LM Studio slightly behind Ollama. All three keep the RTX 3060 under 165 W under sustained generation. Nothing here is stressing the card — 12 GB is comfortable for Q4_K_M weights up to 14B.
Startup time — the underrated metric
| Scenario | llama.cpp | Ollama | LM Studio |
|---|---|---|---|
| Cold start with model loaded | 3.8s | 4.1s | 5.6s |
| Warm start (model in RAM) | n/a | 0.4s | 1.1s |
| Startup after 5min idle | 3.8s | 0.6s (auto-keepalive) | 5.6s |
| First-token latency (200-token prompt) | ~140 ms | ~155 ms | ~180 ms |
Ollama's keepalive daemon is the killer feature here. It keeps the model resident in VRAM across requests, so after the first invocation you get near-instant subsequent responses. llama.cpp's llama-server mode does the same if you leave it running, but you have to leave it running.
VRAM residency — how much do you actually get to use?
| Model | Q4_K_M weights | KV cache @ 8k | Total on GPU | llama.cpp | Ollama | LM Studio |
|---|---|---|---|---|---|---|
| Llama 3.2 3B | 2.0 GB | 0.6 GB | 2.6 GB | 3.1 GB | 3.4 GB | 3.6 GB |
| Mistral 7B | 4.4 GB | 1.0 GB | 5.4 GB | 5.9 GB | 6.2 GB | 6.4 GB |
| Llama 3.1 8B | 4.9 GB | 1.2 GB | 6.1 GB | 6.7 GB | 7.0 GB | 7.2 GB |
| Qwen 2.5 14B | 8.6 GB | 1.8 GB | 10.4 GB | 11.1 GB | 11.3 GB | 11.5 GB |
The "on GPU" column is the theoretical floor; the three actual columns include allocator overhead and framework-side buffers. Note how tight things get at 14B on 12 GB — you have roughly 500 MB of headroom before the driver refuses to allocate. This is exactly why the RTX 3060 12GB is the floor, not the 3060 8GB.
Feature matrix
| Feature | llama.cpp | Ollama | LM Studio |
|---|---|---|---|
| CLI-only inference | yes | yes | no (chat UI required for local) |
| GUI chat | no | no (via Open WebUI) | yes |
| OpenAI-compatible REST API | server mode | yes | yes |
| Model registry with pull commands | manual | yes | yes (built-in browser) |
| GGUF Q4_K_M | yes | yes | yes |
| GGUF Q5_K_M / Q6_K / Q8_0 | yes | yes | yes |
| GPU offload split | fine-grained | auto | auto |
| Multi-GPU | yes | yes | yes |
| Structured JSON output | grammar files | via API | via API |
| Function-calling harness | manual | yes | yes |
| Runs on CPU-only | yes | yes | yes |
| Cross-platform | yes | yes | yes |
| License | MIT | MIT | proprietary (free tier) |
Grammar-constrained decoding is llama.cpp's superpower — you can force JSON, force a specific enum, force a regex. Ollama's structured-output support is improving but is not as flexible. LM Studio inherits llama.cpp's grammar support through the underlying engine but does not expose it as a first-class UI feature.
The friendly-defaults problem
Ollama and LM Studio ship with sane defaults that Just Work for chat. That is a feature. It is also a footgun the moment you want to change them. Ollama's default context length is 2048 tokens on most models — a startlingly small number in 2026. LM Studio defaults to a 4096 context. Both cap KV cache to something safe rather than to what the RTX 3060 12GB can actually hold.
If you notice your local Qwen 14B "forgetting" halfway through a long conversation, check the context setting. On llama.cpp you set --ctx-size 8192 explicitly and take responsibility for the KV memory. On Ollama you set OLLAMA_NUM_CTX=8192 or a Modelfile parameter. LM Studio has a slider buried in the model config page.
Real-world numbers: which one for which workflow?
- Scripting an agentic loop → llama.cpp's
llama-serverwith an HTTP client. Every millisecond of latency matters when you are making 40 tool calls; the extra 8–12% throughput compounds. - Daily driver, VS Code extension, occasional chat → Ollama. The keepalive daemon and REST API are worth the 10% throughput hit.
- Trying out a new model, exploring quants, giving a demo → LM Studio. The GUI is genuinely nice. Nothing else lets you preview a model card, download it, and start chatting inside 90 seconds.
- Grammar-constrained JSON, function-calling, structured extraction → llama.cpp directly, or through a thin OpenAI-compatible wrapper.
Common pitfalls
- Assuming all three use identical tokenization. They do — but Ollama's model-registry versioning can lag Hugging Face by a few weeks. If a new model just landed, pull the raw GGUF for llama.cpp first.
- Running LM Studio in the background. It is not a headless service. If you close the window, the server stops.
- Not adjusting Ollama's context length. Its 2048-token default has bit every developer at least once.
- Fighting VRAM at 14B. If your KV cache OOMs, drop to Q4_0 or reduce context — do not switch cards.
- Assuming faster is always better. LM Studio's slower defaults keep the RTX 3060 cooler and quieter over long sessions. In a fanless-case home-office setup that matters.
When NOT to use any of them
If you need to serve concurrent requests to multiple users, or run production traffic through your local model, vLLM is a better fit than any of the three above. If you are running on Apple Silicon exclusively, MLC-LLM's Metal path outperforms llama.cpp's Metal backend on some models. And if you want the model quantized on the fly with your own calibration data, use llama.cpp's tooling directly — Ollama and LM Studio ship pre-quantized only.
Related guides
- Panther Lake NPU vs RTX 3060 12GB for Local LLM Inference
- OpenAI Codex Record-Replay on a Local Agentic Rig
- Best Budget SATA SSD for a Retro PC Build
