IPEX-LLM + Ollama on Intel Arc: Setup, tok/s, and the RTX 3060 Reality Check

You install IPEX-LLM's Ollama-compatible build, point it at your Arc GPU via the SYCL runtime, and pull a GGUF model — start-to-first-token in about an evening on Linux, longer on Windows. On an Arc B580 you can expect roughly 45 tok/s on Llama 3.1 8B q4, comparable to a well-configured RTX 3060 12GB. The catch: setup is meaningfully more finicky than CUDA, and version-pinning is non-optional.

Why Intel's IPEX-LLM matters for budget local inference

For most of the LLM era, "run a model locally" and "install CUDA drivers" have been the same sentence. IPEX-LLM is the Intel-shipped stack that changes that on Arc. It's a set of PyTorch-XPU accelerations plus a patched Ollama fork plus a llama.cpp SYCL backend, all pinned to matching versions of Intel's oneAPI runtime.

The consumer question is straightforward: does IPEX-LLM close enough of the CUDA gap on cheap Intel silicon to make Arc worth the setup pain? Community measurements say the throughput gap is small. The maintenance gap — driver updates, kernel-support lag on brand-new architectures, occasional silent CPU fallback — is real but manageable.

This is not an academic decision. The MSI RTX 3060 Ventus 3X 12G has held the "cheapest usable local-LLM card" title for two years running because CUDA works out of the box and 12GB is enough for 7B–14B models at q4. IPEX-LLM's job is to make the Arc B580 (also 12GB, ~$249 street) or Arc Pro B60 24GB (~$549) competitive alternatives instead of curiosity purchases.

The rest of this piece walks through what IPEX-LLM is, how to install its Ollama build step by step, what tok/s to expect on 7B / 14B / 32B models versus a CUDA-baseline RTX 3060, and the gotchas that trip most first-time Arc-on-Ollama users.

Key takeaways

• What it is: Intel's LLM stack — patched Ollama + llama.cpp SYCL + PyTorch-XPU + pinned oneAPI runtime. Supports Arc discrete and select Core Ultra iGPUs.
• How fast: ~45 tok/s on Llama 3.1 8B q4 on Arc B580; ~48 tok/s on Arc Pro B60. Comparable to a well-configured 12GB CUDA baseline.
• Setup cost: Budget an evening for a first-time Linux install. Windows works but expect more edge cases.
• The trap: oneAPI + IPEX-LLM + GPU driver versions must match exactly. Mismatch causes silent CPU fallback and confused benchmarking.
• When it wins: VRAM per dollar (Arc Pro B60 24GB), and long-context 14B workloads a 12GB CUDA card can't host cleanly.

What is IPEX-LLM and which Intel GPUs does it support?

Support matrix, distilled from Intel's release notes:

If your card is not on this list, don't waste an evening. Everything below assumes an Arc B580 or Arc Pro B60 on Linux (Ubuntu 22.04 or 24.04) with kernel 6.6+.

How to install the IPEX-LLM Ollama build (step by step)

What you'll need:

• Arc B580 or Arc Pro B60 24GB
• 8-core CPU (an AMD Ryzen 7 5800X handles preprocessing without becoming the bottleneck)
• 32GB system RAM minimum for 14B models
• Fast NVMe boot drive — Samsung 970 EVO Plus 250GB is enough for the OS + one model at a time; add a 1TB SATA SSD like Crucial BX500 for the model library
• Motherboard with Resizable BAR enabled in BIOS
• Ubuntu 22.04 LTS or newer

Steps:

1. Install Intel GPU driver and compute runtime. Enable the Intel Graphics PPA and install intel-opencl-icd, intel-level-zero-gpu, level-zero, and intel-media-va-driver-non-free. Reboot.
2. Confirm the card is visible. Run clinfo | grep -i intel and sycl-ls. The Arc device must appear as an available SYCL platform.
3. Install oneAPI base toolkit at the version listed in the current IPEX-LLM release notes — this is the step where mismatches happen. Use Intel's .sh installer, not apt.
4. Source the oneAPI environment. Add source /opt/intel/oneapi/setvars.sh to your shell profile so sycl-ls and downstream tools pick up the right runtime.
5. Install the IPEX-LLM Ollama build. Grab the Portable Zip from the IPEX-LLM releases page for your OS. Extract, then run ./ollama serve from the extracted directory.
6. Verify GPU is in use. Open a second terminal, run intel_gpu_top, then ollama run llama3.1:8b-instruct-q4_K_M. GPU utilization should climb to 90%+ during generation. If it stays near zero, you're on the CPU fallback path — recheck steps 3 and 4.

The whole flow takes 30–90 minutes on a working system, hours if oneAPI version-pinning goes sideways.

How fast is it? Benchmark table

Per public IPEX-LLM measurements and community reports on r/LocalLLaMA, generation throughput at 4K context, 100-token responses:

At 7B and 8B, Arc's memory bandwidth (456 GB/s vs the 3060's 360 GB/s) delivers a small but real edge. At 14B, that gap widens because the 3060 starts hitting KV-cache pressure. At 32B, the 24GB Arc Pro is the only card of the three that stays on-GPU without offloading layers.

Quantization matrix: VRAM + tok/s + quality loss

Using Llama 3.1 8B as the reference on Arc B580:

q4_K_M is the near-universal sweet spot: it fits the largest usable model onto your card, keeps tok/s high, and shows no visible quality loss for typical chat and assistant work.

Prefill throughput on long prompts: SYCL vs CUDA

Generation is memory-bandwidth bound; prefill is compute-bound. Prefill matters for long-context RAG (dumping 8K tokens of retrieved passages, asking a short question). Community measurements at 8K prefill on Llama 3.1 8B q4:

The 3060 is faster at prefill by a small margin — the CUDA kernels have more optimization years behind them. If your workload is long-context Q&A, that gap is felt as ~0.8s longer wait per query. For interactive chat with short prompts, it's imperceptible.

What breaks: common IPEX-LLM gotchas and version pinning

Ranked by frequency in community reports:

1. oneAPI version mismatch. Installing a newer oneAPI than what IPEX-LLM was built against. Fix: use the exact version listed in the release notes for the IPEX-LLM release you downloaded.
2. Missing source setvars.sh in a new shell. GPU inference silently falls back to CPU because the runtime can't find SYCL. Fix: add to .bashrc or your systemd unit's EnvironmentFile.
3. Resizable BAR disabled. Older AM4 boards ship it off. Performance craters vs the numbers above. Fix: enable in BIOS, reboot.
4. Old kernel without full Xe support. Ubuntu 20.04's default kernel is too old. Fix: use 22.04 LTS or newer.
5. Windows path assuming Linux instructions. Windows works, but many blog guides assume Linux. Use Intel's Portable Zip for Windows and the Windows-specific driver installer.
6. New model architecture not yet in IPEX-LLM. A model that dropped last week may not have SYCL kernel coverage yet. Fix: wait 2–6 weeks or run it on the CPU fallback until the next IPEX-LLM release.

Every one of these produces the same symptom — low tok/s or CPU-level throughput — so it's easy to blame Arc when the fix is a version pin.

When NOT to take this path

Skip IPEX-LLM entirely if any of the following describe you:

• You want tools that "just work." The RTX 3060 12GB CUDA path takes twelve minutes to a working Ollama; IPEX-LLM takes an evening on a good day and a weekend when something snags. If your time is worth even $30/hour, the price gap is repaid before you finish setup.
• You benchmark models the day they drop. New attention variants and architecture tweaks show up in CUDA GGUF loaders within days. SYCL kernel coverage often lags by weeks. If day-one is important, stay on NVIDIA.
• You already own an NVIDIA card that works. The perf uplift from switching to Arc B580 at 8B q4 is small (about 3 tok/s). Not worth reflashing your stack for.
• You're building a headless server that must stay up. IPEX-LLM ships breaking changes across minor versions occasionally. Pin your image, but expect to babysit upgrades more than a CUDA container would need.

The Arc path pays off when the VRAM per dollar advantage matches your workload (14B long-context, 32B at all) and you enjoy tuning the stack. Otherwise it's a hobby, not a shortcut.

Perf-per-dollar vs the RTX 3060 12GB CUDA baseline

Using 8B q4 as the reference workload (45 / 48 / 42 tok/s respectively):

The B580 wins raw perf-per-dollar. The 3060 wins time-to-productivity. The Arc Pro B60 wins if — and only if — you actually need 24GB.

Bottom line: who should take the Arc path

Take the Arc path if:

• You want to run 32B models locally without spending $1,500+ on a used RTX 3090
• You're comfortable pinning versions and reading kernel logs
• You care about long-context 14B workloads more than day-one new-model support

Stay on the RTX 3060 12GB CUDA path if:

• You want the assistant working today, not next weekend
• You track brand-new model releases and want them running the day they drop
• You value framework compatibility (custom nodes, extensions, tooling) over raw VRAM per dollar

Pair either card with a decent CPU like the AMD Ryzen 7 5800X and enough NVMe storage to hold your model library on fast media — model swapping is much less painful when reads land near a gigabyte per second.

Related guides

• Intel Arc B580 & Arc Pro B60 vs RTX 3060 for Local LLMs
• llama.cpp vs Ollama on an RTX 3060 12GB
• vLLM vs llama.cpp on a 12GB GPU
• Best NVMe SSD for Local LLM Model Storage in 2026
• Best GPU for Local LLMs Under $400

Citations and sources

• Intel oneAPI overview
• TechPowerUp — RTX 3060 12GB spec sheet
• Phoronix — Intel Arc B580 compute review

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.