IPEX-LLM + Ollama in Docker on Intel Arc: The 2026 Setup Guide

To run Ollama on an Intel Arc GPU in 2026 you install Intel's IPEX-LLM Ollama portable build inside a Docker container that exposes /dev/dri to the container, pulls the oneAPI base image with SYCL runtime, mounts your model cache, and starts the ipex-llm-ollama binary. That gets you a drop-in Ollama-compatible endpoint on port 11434 that talks to your Arc GPU through Intel's SYCL-backed llama.cpp fork. Total time from bare-metal Ubuntu to first token is ~40 minutes if you follow the recipe below.

Who this guide is for

You have an Intel Arc A770, A750, B580, or Arc Pro B60 and you want to serve local LLMs from a persistent, restartable process rather than a shell you can't close. You want it in Docker so you can reproduce it, ship it to a friend, or move it between hosts without re-doing the driver dance. You've probably tried the stock ollama binary and discovered it silently falls back to CPU on Intel silicon.

The path forward is IPEX-LLM, Intel's inference stack that targets Arc through oneAPI. It ships an Ollama-compatible portable build that speaks the same HTTP API as mainline Ollama. Once running, tools like Continue.dev, OpenWebUI, LangChain's Ollama binding, and the Ollama CLI just work — nothing else changes.

Key Takeaways

• IPEX-LLM Ollama is API-compatible with mainline Ollama; existing clients don't need code changes.
• Docker is the sane distribution channel because it bundles the oneAPI runtime, the SYCL-enabled llama.cpp fork, and the ipex-llm patched Ollama binary in one image.
• You must pass /dev/dri from host to container and grant the render and video groups. Missing that step is the #1 reason first-time setups run on CPU.
• Model weights live outside the container in a bind-mounted directory so restarts don't re-pull. A Crucial BX500 1TB SATA SSD holds a serious model library without stretching your NVMe.
• Expect model releases to lag mainline Ollama by 1-3 weeks; the IPEX-LLM team back-ports upstream Modelfile support on their own cadence.
• Two runtime knobs matter: IPEX_LLM_QUANTIZE_KV_CACHE=1 (~40% VRAM saving for long context) and SYCL_DEVICE_FILTER (pin the GPU when you have multiple Intel devices).

What you need before you start

• A host with a supported Intel Arc GPU. Verified working: A770 16GB, A750 8GB, B580 12GB, Arc Pro B60 24GB. Some Iris Xe integrated GPUs also work with sharply reduced throughput.
• Ubuntu 22.04 LTS or 24.04 LTS on the host. Intel's oneAPI drivers ship best on these; other distros work but need extra kernel-driver work.
• Intel graphics drivers current enough to expose the compute-runtime interface. Version i915 with compute-runtime 24.35+ has been stable.
• Docker 24+ with the docker CLI, buildkit enabled, and non-root access.
• A modest CPU host — a Ryzen 5 5600G is enough for the machine's non-GPU work; a Ryzen 7 5800X is comfortable if you also plan to game or transcode on the same box.
• At least 500 GB of storage for model weights. A Crucial BX500 1TB is the cheap safe path; NVMe is nicer but not necessary because model load is a one-time cost.

If you want CUDA fallback in the same house for models that never made it to IPEX-LLM, an MSI RTX 3060 Ventus 2X 12G plugged into the same host at a different PCIe slot lets you switch between Ollama-CUDA and Ollama-IPEX-LLM by pointing to different endpoints. Overkill for most; occasionally useful.

Step 1 — verify the GPU on the host

Before you touch Docker, confirm your Arc GPU is visible to the kernel and the compute runtime.

sudo apt-get install -y intel-gpu-tools clinfo
lspci | grep -i vga
sudo intel_gpu_top # should show the Arc GPU
clinfo -l # should list an Intel GPU platform + device

If clinfo -l doesn't list your Arc, your compute-runtime install is incomplete. Follow the Intel oneAPI base toolkit install page for your distro before continuing.

Step 2 — grant your user access to the render group

The render group owns /dev/dri/*, and Docker containers inherit host GID mappings for these device files. Add your user (and the container user) to the render and video groups on the host:

sudo usermod -aG render,video $USER
newgrp render

Log out and back in if groups doesn't show render right away.

Step 3 — pull the IPEX-LLM Docker image

Intel ships an official IPEX-LLM Ollama image on Docker Hub as intelanalytics/ipex-llm-inference-cpp-xpu. It bundles the oneAPI base runtime, SYCL support, the ipex-llm Ollama binary, and llama.cpp SYCL. Pull it:

docker pull intelanalytics/ipex-llm-inference-cpp-xpu:latest

The image is heavy — 8-11 GB depending on tag. Grab a tea.

Step 4 — set up your model cache directory

Weights should live outside the container so they survive image upgrades and container recreations:

mkdir -p ~/ollama-models

If you're storing weights on a SATA SSD like the Crucial BX500 1TB or Samsung 870 EVO 250GB, mount it at a stable path (/mnt/models for example) and use that as the bind source. Weights are cold-loaded once per model on server start, so SATA is fine — you'll rarely feel the sequential-read difference versus NVMe.

Step 5 — start the Ollama server in Docker

The critical flags: --device /dev/dri (Arc access), --group-add=render (permission), -v (model cache), and -e OLLAMA_HOST=0.0.0.0 (bind to all interfaces so clients on the LAN can reach it).

docker run -d \
 --name ollama-arc \
 --device /dev/dri \
 --group-add=render \
 --group-add=video \
 -v ~/ollama-models:/root/.ollama/models \
 -p 11434:11434 \
 -e OLLAMA_HOST=0.0.0.0 \
 -e IPEX_LLM_QUANTIZE_KV_CACHE=1 \
 -e SYCL_DEVICE_FILTER=gpu \
 intelanalytics/ipex-llm-inference-cpp-xpu:latest \
 ollama serve

Confirm it started:

docker logs ollama-arc | tail -20
curl http://localhost:11434/api/version

Expected: a JSON blob with a version string. If curl fails, the server is either not listening on 0.0.0.0 or the container died — check docker ps and docker logs.

Step 6 — pull a model and generate

The IPEX-LLM Ollama binary speaks the same /api/pull and /api/generate API as upstream Ollama:

docker exec ollama-arc ollama pull qwen3:14b
docker exec ollama-arc ollama run qwen3:14b "Explain SYCL in one paragraph."

First-token latency runs 500-900 ms on a warm model. Steady-state generation for Qwen 3 14B q4_K_M on an Arc A770 16GB lands around 30 tok/s.

Step 7 — expose it to clients

At this point the container is a full Ollama endpoint on http://<host-ip>:11434. Point Continue.dev, OpenWebUI, or your LangChain client at that URL. The local LLM in VSCode guide walks through the Continue.dev side.

For OpenWebUI:

docker run -d --name openwebui \
 -p 3000:8080 \
 -e OLLAMA_BASE_URL=http://<host-ip>:11434 \
 ghcr.io/open-webui/open-webui:main

Browse to http://localhost:3000, register, done.

Runtime knobs that matter

• IPEX_LLM_QUANTIZE_KV_CACHE=1: q8-quantizes the KV cache. ~40% VRAM savings on long contexts, ~1-3% quality tax, negligible throughput hit.
• SYCL_DEVICE_FILTER=gpu: pins execution to a discrete GPU rather than iGPU or CPU. Set explicitly in multi-GPU systems.
• OLLAMA_NUM_PARALLEL=1: leave at 1 for chat. Set to 2-4 only if you have VRAM headroom and want concurrent request handling.
• OLLAMA_MAX_LOADED_MODELS=1: prevents multiple models sharing VRAM simultaneously.

Reported performance across models

Numbers below are reported community results for IPEX-LLM Ollama on Arc A770 16GB at batch-1, 128-token prompt, 512-token generation.

On the B580 12GB, subtract 3-5 tok/s. On the Arc Pro B60 24GB, add 2-4 tok/s and extend the model list to include 32B at q4_K_M — see the Arc Pro B60 guide.

Troubleshooting

Container starts but generation is CPU-slow. The container isn't seeing the GPU. Check docker exec ollama-arc clinfo -l — you should see an Intel GPU device. If it lists only CPU, the --device /dev/dri and --group-add=render flags aren't taking effect. Rebuild the container.

"unsupported model" errors on new upstream models. IPEX-LLM lags mainline. Wait for the next ipex-llm-inference-cpp-xpu image or check the IPEX-LLM GitHub issues for the model-support tracker.

OOM on a model that fits weights. KV cache with a big context blew the budget. Set IPEX_LLM_QUANTIZE_KV_CACHE=1 and reduce your num_ctx in the Modelfile from 32K to 16K or 8K.

Slow prefill / first-token time. Arc's prompt-processing throughput trails CUDA. If your prompts are 8K+ tokens, expect 2-4 seconds of prefill time before the first output token. That's a stack issue, not a config problem.

Docker complains about /dev/dri permission. Add the render and video GIDs from your host to the container: --group-add=$(getent group render | cut -d: -f3) --group-add=$(getent group video | cut -d: -f3).

Persistence, restarts, and updates

The setup above uses docker run -d which restarts the container on docker start. For a permanent service, add a --restart=unless-stopped flag or wrap it in a systemd unit that runs docker start ollama-arc. Model weights persist in ~/ollama-models across container recreations. Image updates are docker pull intelanalytics/ipex-llm-inference-cpp-xpu:latest && docker stop ollama-arc && docker rm ollama-arc && <re-run the docker run command above>.

What NOT to do

• Don't run stock upstream Ollama and expect it to see your Arc GPU. It won't. The CUDA build silently falls back to CPU on non-NVIDIA hardware.
• Don't try to run this on WSL2 without configuring /dev/dri passthrough — that's a longer setup and gains you nothing over a native Linux boot.
• Don't run IPEX-LLM Ollama on the host directly if you also want to run mainline Ollama for CUDA. They'll fight over the same port and model directory.

Bottom line

IPEX-LLM Ollama in Docker is the least fragile way to run Ollama on an Intel Arc GPU in 2026. The install is 20 minutes of setup and gives you an Ollama-compatible endpoint that survives restarts, migrates cleanly between hosts, and lets your existing Ollama tooling work unchanged. What you give up is mainline-day-one support for new models; what you get is a real, deterministic runtime on Intel silicon.

Related guides

• Intel Arc B580 for Local LLMs in 2026
• Intel Arc Pro B60 24GB AI Inference Guide
• Local LLM in VSCode: The 2026 Setup Guide
• How to Run Qwen 3 14B on Arc B580
• Building a Budget Local-AI Box: Ryzen 7 5800X + RTX 3060 12GB

Frequently asked questions

Do I need Intel's Data Center GPU version of the driver, or the consumer one? Use the consumer Intel Graphics driver stack. IPEX-LLM's Docker image supports both, but the consumer driver is easier to install, well-documented for Ubuntu, and works with all Arc A- and B-series cards. The Data Center GPU driver is for Flex/Max/Ponte Vecchio hardware, not consumer Arc.

Can I run multiple models concurrently on one Arc GPU? Yes, if VRAM allows and you set OLLAMA_MAX_LOADED_MODELS=N. Practical: on a 16GB Arc A770 you can hold one 14B and one 7B model resident simultaneously. Requests queue per model. Concurrent inference across two models cuts effective throughput on each because compute is shared.

How does this compare to running llama.cpp SYCL directly without Ollama? Direct llama.cpp gives you slightly more control (custom sampler settings, exotic quantizations, RPC serving) at the cost of losing Ollama's model-management convenience (ollama pull, Modelfile). For an interactive chat setup, Ollama's ergonomics win. For a serving pipeline, direct llama.cpp with a custom Python or Rust wrapper is sometimes cleaner.

Does IPEX-LLM Ollama support GGUF and llama.cpp Modelfile syntax? Yes. IPEX-LLM's Ollama fork ingests GGUF models directly and understands the standard Modelfile syntax including FROM, PARAMETER, TEMPLATE, SYSTEM, and adapters. Some newer parameters that upstream Ollama ships early may take 1-2 weeks to reach the IPEX-LLM build.

Can I run this on a headless server without a display? Yes. /dev/dri doesn't require an attached display — the render node is separate from the display node. Confirm with ls /dev/dri — you want renderD128 (or a similar renderD*) present. On truly headless installs you may need to blacklist the framebuffer driver or add nomodeset to grub, but the render node stays available.

Citations and sources

• Intel IPEX-LLM GitHub repository
• Intel oneAPI base toolkit overview
• Phoronix — Intel IPEX-LLM news coverage

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.