How to run Qwen 3 32B on Apple M4 Pro

Qwen 3 32B needs the 48 GB or 64 GB M4 Pro, and it tops out near 14 tokens per second there. Qwen's official Q4_K_M GGUF is 19.76 GB, which leaves a 24 GB M4 Pro almost nothing for macOS and the context cache. Apple lists the M4 Pro with up to 64GB of unified memory and 273GB/s of memory bandwidth. Because a dense 32B model reads all its weights for every token, 273 GB/s divided by 19.76 GB caps generation at about 14 tok/s, and real runs land below that.

Correction (2026-09-26). An earlier version of this page quoted 20–28 tok/s for Q4_K_M on an M4 Pro, from a test setup SpecPicks never ran. Those figures exceed what the chip's memory bandwidth allows and have been removed.

Quick answer: file size, fit and speed ceiling per quant

The ceiling column is 273 GB/s (Apple) divided by the file size: arithmetic, not a measurement. The KV cache grows with context and needs room on top of the file size, and macOS keeps part of unified memory for itself.

Which M4 Pro to buy for it

• 24 GB (base). Not for this model. The Q4_K_M weights alone are 19.76 GB. The Mac mini M4 Pro in our catalog is this 24 GB configuration, so it suits the smaller Qwen 3 14B instead.
• 48 GB. The practical choice. Q4_K_M through Q6_K fit with room for a long context; Q8_0 is tight.
• 64 GB. The top M4 Pro configuration per Apple. Every quant fits, including Q8_0, with room left for other apps. More memory does not raise the speed ceiling: bandwidth is the same 273 GB/s.

Higher quants trade speed for quality: each step up makes the file larger and the ceiling lower.

Step 1 — Install Ollama

curl -fsSL https://ollama.com/install.sh | sh
ollama --version

The Ollama installer sets up the launch agent and the Metal backend. No CUDA drivers, no Docker.

Step 2 — Pull and run Qwen 3 32B

ollama pull qwen3:32b
ollama run qwen3:32b

The Ollama library lists the qwen3:32b download at 20GB.

By default Qwen 3 emits a <think> reasoning trace before each answer. To suppress it for chat-first work:

ollama run qwen3:32b "/no_think Write a Bash script to rotate log files older than 30 days."

With thinking on, the model writes its reasoning before the answer, so the wait for an answer grows even though the per-token speed does not change.

Step 3 — llama.cpp for more control

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release -j 8

./build/bin/llama-server \
 -m Qwen3-32B-Q4_K_M.gguf \
 -c 16384 -ngl 999 -fa --mlock \
 --host 0.0.0.0 --port 8080

llama.cpp builds with Metal on Apple silicon by default. -ngl 999 puts every layer on the GPU, and -fa turns on flash attention, which cuts attention memory use at long context. The llama.cpp Apple silicon performance thread collects community results across M-series chips if you want to compare your numbers.

Step 4 — Measure your own speed

# Prints prompt-eval and eval (decode) rates after the answer
ollama run qwen3:32b --verbose "/no_think Hello"

# Memory pressure (keep it green during inference)
memory_pressure

If the eval rate is far below the ceiling in the table, check memory pressure first: when macOS starts compressing or swapping, generation slows sharply. Shorter context, a smaller quant, or closing other apps are the usual fixes.

Sample Modelfile recipes

# qwen3-32b-fast — chat, no thinking, low context
FROM qwen3:32b
PARAMETER num_ctx 4096
PARAMETER num_predict 1024
PARAMETER temperature 0.7
SYSTEM """You are a fast assistant. Never emit <think> blocks unless asked."""

# qwen3-32b-reasoning — full thinking mode, code/math
FROM qwen3:32b
PARAMETER num_ctx 16384
PARAMETER num_predict 4096
PARAMETER temperature 0.6
SYSTEM """You are a careful reasoner. Show your work in <think> blocks before answering."""

Bottom line

Buy the 48 GB M4 Pro (or 64 GB for Q8_0 and headroom) and expect under 14 tok/s at Q4_K_M, because memory bandwidth, not memory size, sets the speed. If you need a faster 32B, you need more bandwidth: a higher-bandwidth chip or a desktop GPU with 24 GB or more. For which open models fit which GPUs, see Will it run?.

Citations and sources

• Qwen — Qwen3-32B model card: model details.
• Qwen — Qwen3-32B-GGUF: the Q4_K_M, Q5_K_M, Q6_K and Q8_0 file sizes.
• Ollama — qwen3 library: the 20GB qwen3:32b download.
• Apple — M4 Pro and M4 Max announcement: up to 64GB unified memory, 273GB/s memory bandwidth.
• llama.cpp and its Apple silicon performance thread: runtime and community results.

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.

Products mentioned

• Apple 2024 Mac mini with M4 Pro (24 GB configuration)