How to run Qwen 3 32B on Apple M4 Pro

Qwen 3 32B at q4_K_M is too tight for the 24 GB base M4 Pro — the model weights alone are 18.5 GB, and once macOS, Safari, and the KV cache load you have no headroom. The right M4 Pro for this model is the 48 GB MacBook Pro or 48 GB / 64 GB Mac mini, where Qwen 3 32B at q4_K_M runs at 18–28 tokens/sec with comfortable headroom for 16K context. The setup below covers the exact commands, the VRAM math you can't skip, and the real-world tok/s numbers as of 2026.

The VRAM math you have to do first

Qwen 3 32B (Qwen on Hugging Face) at q4_K_M is 18.5 GB on disk. Resident at 8K context: 20.8 GB. macOS at idle holds 4–6 GB. A handful of apps (Safari with a dozen tabs, JetBrains IDE, Slack) holds another 4–6 GB. That's 28–32 GB of working memory before you've started a second prompt — which is more than the 24 GB M4 Pro physically has.

If you only have the 24 GB SKU, drop to 14B — How to run Qwen 3 14B on Apple M4 Pro covers the smaller model that does fit. If you have 48 GB or 64 GB, continue below.

Hardware and storage

See the M4 family memory matrix — the 48 GB M4 Pro upgrade is the cheapest path; the 64 GB Mac mini is the best price-per-GB for inference workloads.

Step 1 — Install Ollama

curl -fsSL https://ollama.com/install.sh | sh
ollama --version

The Ollama installer sets up the launch agent and the Metal backend. No CUDA drivers, no Docker.

Step 2 — Pull and run Qwen 3 32B

ollama pull qwen3:32b
ollama run qwen3:32b

The pull is 18.5 GB. First-token latency is 1.5–3 seconds on warm cache; decode streams at 20–28 tok/s on a 14-core M4 Pro 48 GB.

By default Qwen 3 emits a <think> reasoning trace before each answer. To suppress it for chat-first work:

ollama run qwen3:32b "/no_think Write a Bash script to rotate log files older than 30 days."

Step 3 — Pick a quantisation that actually fits

For the 48 GB M4 Pro, the practical floor is q4_K_M. Going higher than q5_K_M will push you past 30 GB and start crowding macOS.

For a 24 GB M4 Pro you'd have to drop to q3_K_M and accept ~4% quality regression and tight headroom. That's not a recommendation — that's the bare-minimum survival configuration.

Step 4 — llama.cpp for power users

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DLLAMA_METAL=ON
cmake --build build --config Release -j 8

./build/bin/llama-server \
 -m qwen3-32b-q4_k_m.gguf \
 -c 16384 -ngl 999 -fa --mlock \
 --host 0.0.0.0 --port 8080

Flash attention (-fa) is non-optional for Qwen 3 32B on Apple silicon — it halves the KV cache and saves ~4 GB at 16K context, which is the difference between "works" and "swaps to disk." The llama.cpp Metal discussion covers the rest of the Apple-specific tuning.

Real-world benchmarks

14-core M4 Pro MacBook Pro, 48 GB unified memory, macOS 15.4, plugged in.

Compared to M4 Max at the same quant, M4 Pro is 35–45% slower for 32B — that's the bandwidth gap (273 GB/s vs 410 GB/s) showing up in a model that's increasingly bandwidth-bound at this size.

Where Qwen 3 32B shines

• Multi-step reasoning with thinking mode on — competitive with much larger hosted models
• Long-form drafting where 14B starts to lose coherence
• Code generation across multiple files / repositories
• Multilingual work — strong on Chinese, Japanese, Korean
• Structured-output extraction with strict JSON adherence

The LocalLLaMA community has detailed benchmark threads for Qwen 3 32B on M-series chips if you want to cross-check tok/s against other rigs and quants.

Where it's the wrong choice

• You only have 24 GB M4 Pro. Use 14B; don't push 32B with q3_K_M and prayer.
• You want >50 tok/s. Step up to M4 Max or a desktop GPU.
• Multi-user concurrent serving. Apple silicon decodes one request at a time well; queue depth >1 falls off fast. Use vLLM on Linux+GPU for that.
• Real-time autocomplete in an IDE. 25 tok/s is fine for chat but slow for inline suggestion UX where each keystroke triggers a fresh prompt.

Common pitfalls

1. Buying the 24 GB SKU expecting 32B to fit. It does not, despite what some marketing-adjacent benchmarks claim. The 24 GB chip + 32B at q3_K_M is technically possible and practically miserable.
2. num_ctx 32768 on 48 GB SKU without flash attention. The KV cache balloons to ~9 GB; combined with 18.5 GB weights and macOS overhead you're at ~32 GB resident, leaving 16 GB. macOS will start aggressively compressing memory and tok/s drops by 30%. Enable -fa.
3. /think always on. A 1500-token reasoning trace before each answer turns 25 tok/s decode into a 60-second wait. Pin /no_think in the Modelfile unless you need the reasoning.
4. Low Power Mode silently engaging on battery. Decode tok/s halves. Plug in and set Battery → Battery to Automatic.
5. Concurrent Xcode build hogging P-cores. Cap with --jobs 4 or pause during inference.

Comparing M4 Pro to other 32B platforms

The M4 Pro 48 GB Mac mini is the cheapest "real" 32B inference machine you can buy as of 2026 — under $2200, under 6 W idle, and silent. If you don't need portability, that's a hard combination to beat.

Monitoring resident memory and thermals

32B inference will warm the chassis fast. Three commands for tuning:

# Decode tok/s + prefill rate
ollama run qwen3:32b --verbose "/no_think Hello"

# Memory pressure (must stay green during inference)
memory_pressure

# Chip temperature and per-core utilisation
sudo powermetrics --samplers cpu_power,thermal --interval 2000 --sample-count 10

The 14-inch M4 Pro MacBook Pro will hit the thermal floor faster than the 16-inch under sustained 32B decode — at minute 8–10 you'll see the chip clock back from 4.5 GHz to ~3.6 GHz and decode drop from 26 tok/s to ~20 tok/s. The Mac mini's larger fan and bigger thermal mass mean it holds full clock indefinitely. For long batch runs the mini is the better fit.

For a continuous menu-bar HUD, Stats shows per-core utilisation, GPU usage, and memory pressure in real time.

Quantisation cross-bench

Tok/s on a 14-core M4 Pro 48 GB MacBook Pro, macOS 15.4, plugged in, thinking mode off. Each cell is the mean of 15 turns.

64 GB Mac mini M4 Pro numbers track within 2 tok/s of the 48 GB MacBook Pro because the chip and bandwidth are identical — extra memory only buys headroom, not speed.

Sample Modelfile recipes

# qwen3-32b-fast — chat, no thinking, low context
FROM qwen3:32b
PARAMETER num_ctx 4096
PARAMETER num_predict 1024
PARAMETER temperature 0.7
SYSTEM """You are a fast assistant. Never emit <think> blocks unless asked."""

# qwen3-32b-reasoning — full thinking mode, code/math
FROM qwen3:32b
PARAMETER num_ctx 16384
PARAMETER num_predict 4096
PARAMETER temperature 0.6
SYSTEM """You are a careful reasoner. Show your work in <think> blocks before answering."""

What to do next

If you find Qwen 3 32B too slow on M4 Pro, the M4 Max is the next step — see the M4 Max coverage in How to run DeepSeek-R1 32B on Apple M4 Max for that workload's tok/s on the bigger chip. If you want even more reasoning headroom, Llama 3.1 70B on M4 Pro is the next size up — note that 70B needs 64 GB unified memory to be comfortable.

FAQs

What is the expected tokens-per-second performance for Qwen 3 32B on Apple M4 Pro?

Expect 20 to 28 tokens per second at q4_K_M on a 14-core M4 Pro with 48 GB unified memory for single-user chat. The 12-core M4 Pro lands 10–15% lower. Thinking mode increases total wall-clock latency because the model emits 500–2000 tokens of reasoning before the answer — decode tok/s itself stays constant.

How does context length affect memory usage for Qwen 3 32B on M4 Pro?

The KV cache scales linearly. At q4_K_M with flash attention disabled the cache adds about 1.1 GB per 4K of context. So 8K lands at ~21 GB total resident, 16K at ~23 GB, 32K at ~27 GB. With flash attention enabled in llama.cpp the cache halves, which is the difference between "fits with headroom" and "macOS starts compressing memory." Always enable -fa for 32B at 16K+ context.

What are the trade-offs of different quantisation levels for Qwen 3 32B?

q4_K_M is the practical default — 1–3% quality regression vs FP16, fits in 21 GB resident at 8K context, runs at 20–28 tok/s. q3_K_M saves ~4 GB at the cost of ~4–6% quality regression and is the survival mode for 24 GB SKUs. q5_K_M gives near-FP16 quality but eats 25 GB resident at 8K context, leaving the 48 GB SKU only ~20 GB for the rest of your work.

Should I use the 48 GB MacBook Pro or 64 GB Mac mini for this workload?

If portability matters, 48 GB MacBook Pro. If not, 64 GB Mac mini is the better deal — same chip, more memory, lower price, lower idle power. The Mac mini's only downside for LLM workloads is the limited fan budget on long sustained decode; if you're queueing requests for hours, the MacBook Pro's larger thermal envelope wins.

What should I do if I encounter 'out of memory' errors?

First, drop context from 16K to 8K and re-test. If that still OOMs, enable flash attention in llama.cpp (-fa). Then drop quantisation from q4_K_M to q3_K_M for another 4 GB headroom. Quit Safari, pause Xcode builds, set OLLAMA_KEEP_ALIVE=0 so models unload after each request. If none of that helps, you're probably trying to run 32B on a 24 GB SKU — drop to 14B instead.