How to run DeepSeek-R1 32B on Arc B580

This tutorial walks you through running DeepSeek-R1 32B on an Arc B580. Exact commands, expected tokens-per-second, and the tradeoffs you should know before starting.

Does it fit?

Arc B580 has 12 GB of GDDR6. DeepSeek-R1 32B at q4_K_M wants ~19 GB of it for weights alone, plus another 2-3 GB of KV cache at 4K context.

Verdict: ⚠️ Tight fit. You'll need CPU offload (llama.cpp with 5 layers on CPU) or drop to q3_K_S. Expect 15-25 tok/s with offload.

Install Ollama (the easy path)

# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh
ollama pull deepseek-r1:32b
ollama run deepseek-r1:32b

Ollama handles GPU detection automatically on NVIDIA (via CUDA), AMD (via ROCm on Linux), and Apple Silicon (via Metal). For Intel Arc B580, upstream Ollama support uses llama.cpp's SYCL backend and may require a SYCL-enabled build or the IPEX-LLM fork of Ollama — confirm the Arc is detected at startup before assuming it offloaded.

Install llama.cpp (more control)

llama.cpp gives you flag-level control over quantization, context length, and layer offload. Build from source:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j

# Download a quantized GGUF (community favorite: bartowski or TheBloke on HuggingFace)
./llama-cli -m ~/models/deepseek-r1-32b-q4_k_m.gguf \
 -n 512 -c 4096 \
 -ngl 10 \
 -p "Write a haiku about GPUs"

With 12GB VRAM, offload ~10 layers; the rest stays on CPU. Expect roughly half the all-GPU speed but much cleaner than running at smaller quants.

Expected performance

• Community reports from LocalLLaMA suggest ~10-25 tok/s on this class of hardware.

If you need more speed, bigger VRAM is the answer. Consider upgrading to a card with more headroom →.

Common issues

"out of memory" on the first prompt: reduce context length (-c 2048) or quantization (q4_K_S instead of q4_K_M).

Slow first token but fast generation: that's prompt processing ("prefill"). Normal — blame the KV cache building. Subsequent messages in the same session will be snappy.

Frequent swapping / system hangs: VRAM is full AND system RAM is full. Close Chrome. Add more DDR5.

Related

• Full Arc B580 benchmarks →
• Best GPU for DeepSeek-R1 32B →
• AI Rigs buyer's guide →

Arc B580 specs: 12GB memory, 190W TDP, 2024 launch. MSRP $249.

Does it fit? Full quantization matrix

Weight-only VRAM for DeepSeek-R1 32B at every common quant, plus the KV-cache overhead for a 4K-token context. KV cache scales linearly with context — see the context-length table further down.

Values are approximate — actual footprint depends on batch size, whether the KV cache is quantized (-ctk q8_0 -ctv q8_0 in llama.cpp halves it), and whether you reserve VRAM for a display. Rule of thumb: budget 5-10% headroom on top of the table.

How public benchmarks show and compared

Every tok/s, FPS, and synthetic score in this article is pulled live from the SpecPicks benchmark catalog (hardware_specs, ai_benchmarks, synthetic_benchmarks). We cite the source_name on each row — the vast majority are community-reported numbers from r/LocalLLaMA and llama.cpp GitHub Discussions, with synthetic scores from PassMark, Phoronix, and Tom's Hardware's GPU hierarchy.

Where DB rows exist for a specific model+quant+GPU combination, we quote the number exactly. Where they don't, we fall back to published spec-sheet values (VRAM capacity, TDP, memory bandwidth) plus the closest community-verified ballpark — clearly flagged as a ballpark, not a measurement. We prefer "we don't know" over a fabricated number.

SpecPicks does not run paid hardware review cycles; we aggregate. If you see a number you can improve on, pull-request the row.

Measured tok/s on this GPU

Live data from ai_benchmarks for Arc B580, filtered to the DeepSeek-R1 32B family where available:

For the full tok/s matrix on this card across every model we've logged, see the Arc B580 benchmark page.

Context length and VRAM — the hidden cost

KV cache grows linearly with context. Here's the approximate overhead on top of 19.2 GB of q4_K_M weights for DeepSeek-R1 32B:

For long-context workloads (32K and beyond) on consumer hardware, use llama.cpp's KV-cache quantization — -ctk q8_0 -ctv q8_0 roughly halves cache footprint with sub-1% quality loss. This is the single biggest VRAM-saving flag for long context.

Which runtime wins on this hardware?

Three mainstream runtimes target Arc B580; the right one depends on your workload:

• Ollama — easiest. Auto-detects SYCL / Level Zero, handles model downloads, exposes an OpenAI-compatible API out of the box. Wraps llama.cpp; you give up fine-grained control for zero setup.
• llama.cpp — direct flag-level control over quant, context, KV-cache precision, batch size, split layers across GPUs. Where the LocalLLaMA community benchmarks its numbers (see the Apple-Silicon megathread #4167 for reference tok/s across M-series chips).
• vLLM — built for production serving. Tensor parallelism, PagedAttention, continuous batching. Limited support on this platform — Ollama/llama.cpp are safer bets. If you're not serving multiple concurrent users, the overhead isn't worth it.

For head-to-head numbers and install commands across all three, see our Ollama vs llama.cpp vs vLLM guide.

Troubleshooting — three failure modes and fixes

1. First token takes 5-30 seconds, then generation is fast. That's normal prefill: the model is processing your prompt before it can start generating. On a long prompt (4K+ tokens) prefill dominates the first-token latency. If it's unexpectedly slow, check that you actually offloaded layers to the GPU — intel_gpu_top or xpu-smi should show near-100% utilisation during prefill. If utilisation is flat, your inference is running on CPU.

2. "Out of memory" halfway through a long chat. The KV cache grew past what the card can hold. Drop to a smaller quant (q4_K_M → q3_K_M), cut -c context length, or enable KV-cache quantization (-ctk q8_0 -ctv q8_0 in llama.cpp). On Ollama set num_ctx smaller in your Modelfile.

3. Tok/s is ~30% of what LocalLLaMA threads report. Three usual suspects: (a) power/thermal throttling — check sustained clocks during a long prompt; (b) PCIe x8 or x4 link when you expected x16 — lspci -vv | grep -i 'LnkSta:' for the Arc device; (c) running the wrong binary, e.g. a CPU-only llama.cpp build on the Arc. Rebuild with GGML_SYCL=1 (oneAPI) or GGML_VULKAN=1 and confirm SYCL/Vulkan devices are listed at startup.

Sources

1. r/LocalLLaMA (community tok/s threads)
2. llama.cpp GitHub Discussions #4167 — Apple Silicon benchmark thread
3. Tom's Hardware GPU Hierarchy

Related guides

• Arc B580 full benchmark page
• Best GPU for DeepSeek-R1 32B
• Ollama vs llama.cpp vs vLLM
• What VRAM do you need for local LLMs?