oQ vs Q4_K/Q6_K vs MXFP4 vs UD-MLX: Which Quantization Format Should You Pick in 2026?

What changed since April 2026: August 2026 measurements add two qualifiers: kelnei's Qwen3.8-27B NVFP4 card puts the cost of quantizing activations at roughly 2× (mean KL 0.0134 weight-only vs 0.0259 with W4A4 emulated), and on DeepSeek-V4-Flash, whose experts ship in MXFP4, AtomicChat's 8× RTX 5090 KLD ladder has MXFP4 (0.156403) tied with Q4_K_M (0.155595) and UD-Q4_K_XL (0.155725).

As of October 2026, the measured answer is still that no 4-bit format is free. Pick plain 6-bit or oQ6 when memory allows, and use FP4 formats for prompt-processing speed rather than quality. The newest public NVFP4 numbers come from akopytko's Qwen3.8-27B NVFP4 GGUF model card, updated October 4, 2026. Measured against BF16 on wikitext, Unsloth's NVFP4 build has a mean KLD of 0.0708, Unsloth's Q4_0 has 0.0283 and Unsloth's Q6_K has 0.0029. NVIDIA's own NVFP4 checkpoint, rebuilt with Q6_K in place of FP8, scores 0.0393. On an RTX 5090, the same card shows Unsloth NVFP4 processing a 2,048-token prompt at about 6,019 tok/s, against about 4,157 tok/s for Unsloth Q4_0. Decode speed is essentially the same (87.69 vs 87.01 tok/s). On Apple Silicon the ranking has not changed. On Qwen3.6-27B, smcleod.net's 65,536-token KL-divergence test on an M5 Max found mlx-community's plain 6-bit (22.78 GB) at a mean KLD of 0.0287 and oQ6 at 0.0296. Unsloth's UD-MLX-4bit used 26.19 GB (8.6 effective bpw) and scored a worse 0.0592, and oQ4 beat plain 4-bit by only about 3% (0.110 vs 0.113). On the GGUF side, Unsloth's own Qwen3.5-35B-A3B benchmarks put UD-Q4_K_XL at a mean KLD of 0.0137 against 0.0272 for MXFP4_MOE. KLD values from different models and test corpora are not directly comparable; only the ranking within each test carries over.

What changed — October 2026

• NVFP4 GGUF still trails Q4_0 on quality. On Qwen3.8-27B, per akopytko's NVFP4 GGUF model card (updated October 4, 2026), Unsloth's NVFP4 has a mean KLD of 0.070824 vs BF16 with 88.061% top-token agreement. Unsloth's Q4_0 scores 0.028317 (92.698%) and Unsloth's Q6_K scores 0.002857 (97.506%). The card's own conclusion is that NVFP4 "falls behind Q4_0, so no surprises here."
• NVIDIA's own NVFP4 checkpoint lands between them. The October 4 update added a GGUF build of nvidia/Qwen3.8-27B-NVFP4 with Q6_K in place of FP8. It scores a mean KLD of 0.039314, better than Unsloth NVFP4 but still behind Unsloth Q4_0 (model card).
• On Blackwell, NVFP4's advantage is prefill, not decode. On an RTX 5090 at stock 575 W, Unsloth NVFP4 processes a 2,048-token prompt (pp2048) at 6,018.71 tok/s, against 4,156.58 for Unsloth Q4_0. Token generation (tg128) is 87.69 vs 87.01 tok/s (model card).
• W4A4 roughly doubles NVFP4's KL cost. Per kelnei's Qwen3.8-27B NVFP4 model card (August 2026), full-vocab KL against BF16 over 30 chat prompts × 384 tokens is 0.0134 mean with BF16 activations ("weight-only", 96.1% top-1 agreement) and 0.0259 with activations fake-quantized to approximate W4A4 serving (94.1%). The same card reports 55.4 tok/s single-stream without speculation on an RTX PRO 6000 Blackwell in vLLM 0.27.1. Its corpus differs from the wikitext run above, so compare the two results only within each table.
• When a model ships in MXFP4, re-quantizing to Q4_K buys nothing at 4-bit. On DeepSeek-V4-Flash-0731, where "the routed experts of this model are 96% of its weights and they are stored in MXFP4", AtomicChat's shared-reference KLD dataset (August 2026) measured AtomicChat's MXFP4 at 0.156403, DevQuasar's MXFP4_MOE at 0.156042, DevQuasar's Q4_K_M at 0.155595 and Unsloth's UD-Q4_K_XL at 0.155725. The page's stated noise is about ±0.003, so these four are a tie. Only a larger Q4_K-experts mix from antirez (164.6 GB, 0.123713) and Q5_K_M (0.120901) measured clearly lower.

What changed since the April version of this guide

The first version of this article, published in April 2026, ranked UD-MLX first at every bit width and treated MXFP4 as a quality upgrade. The public measurements published since then do not support either claim. The earlier KLD and speed tables could not be traced to a published source, so this revision replaces them with figures that are cited line by line. Three corrections matter most.

First, "oQ vs Q vs MXFP vs UD MLX" is a comparison of MLX formats, not GGUF formats. Q is mlx-lm's plain affine round-to-nearest (RTN) quantization. MXFP is MLX's microscaling float modes; MLX's documented modes are affine, mxfp4, mxfp8 and nvfp4, with no MXFP6. oQ is oMLX's data-driven mixed-precision quantizer, which writes standard MLX safetensors. It is not a calibrated GGUF variant. UD MLX is Unsloth's Dynamic recipe ported to MLX, which Unsloth itself calls "still evolving."

Second, MXFP4 and NVFP4 are different formats. Both store 4-bit E2M1 values. MXFP4 shares one E8M0 scale per 32 weights, while NVFP4 uses 16-weight groups with an E4M3 scale. MLX's docs require a group size of 16 for nvfp4.

Third, the hardware story moved. llama.cpp has shipped an MXFP4 data type since gpt-oss support landed in August 2025 (PR #15091). NVFP4 kernels followed in spring 2026 (dp4a kernel, PR #20644; generic MMQ, PR #21074), and the Blackwell-native NVFP4 path (PR #22196) merged on April 28, 2026.

Key takeaways

• The MLX quality ranking on Qwen3.6-27B (dense) is 8-bit, then 6-bit ≈ oQ6, then UD-MLX-4bit, then oQ4 ≈ plain 4-bit. Per smcleod.net, the mean KLDs are 0.0139, 0.0287, 0.0296, 0.0592, 0.1100 and 0.1132.
• UD-MLX-4bit is 4-bit in name only. It is 26.19 GB on disk, more than the 22.78 GB plain 6-bit, and roughly 2× worse on mean KLD in that test.
• On MoE models, the recipe matters more than the format family. On Qwen3.6-35B-A3B, oQ6 beat plain 6-bit by about 13% (0.0104 vs 0.0119). mlx-community's DWQ 4-bit beat oQ4 by about 34% (0.0266 vs 0.0402), because DWQ protects the router tensors (smcleod.net).
• Within MLX FP4, NVFP4 beats MXFP4 on quality. Unsloth's Qwen3.6-27B MLX table shows NVFP4 (UD) at a mean KLD of 0.0325 against 0.0479 for MXFP4 (UD), at similar sizes (26.2 GB vs 25.6 GB) (Unsloth).
• In GGUF, K-quants beat MXFP4 on quality. Unsloth says "MXFP4 is much worse on many tensors" and is retiring MXFP4 from its Q2_K_XL, Q3_K_XL and Q4_K_XL quants, except pure MXFP4_MOE builds (Unsloth Qwen3.5 benchmarks). The exception is a model released in MXFP4: on DeepSeek-V4-Flash, MXFP4 and Q4_K_M land within noise of each other (0.156403 vs 0.155595 mean KLD, AtomicChat).
• Speed follows bytes read per token. On the same M5 Max, a 16 GB 4-bit Qwen3.6-27B decodes at 32.1 tok/s and the 26 GB UD build at 20.2 tok/s (oMLX community benchmarks).

What is KL divergence and why does it matter more than perplexity?

Perplexity measures how surprised a model is by held-out text. It has been the default quantization metric because it is cheap to compute. Its weakness is that it mixes the model's own uncertainty with the damage done by quantization, so two quants can post near-identical perplexity while their output distributions differ.

KL divergence (KLD) compares the quantized model's full next-token distribution against a reference model (usually BF16) at every position, then averages. Zero means the predictions are identical. smcleod.net's scale for Qwen-class models puts roughly 1e-3 to 5e-3 at "typical of well-made 6-bit" and 1e-2 to 5e-2 at "typical 4-bit territory." Anything above 1e-1 counts as substantial divergence, where sampled outputs likely differ obviously.

Perplexity misses real gaps in the published numbers. In Unsloth's Qwen3.6-27B MLX table, the 8-bit, 6-bit and UD 4-bit builds post perplexities of 4.812, 4.809 and 4.821, a spread under 0.3%. Their mean KLDs are 0.0028, 0.0037 and 0.0227, a 6-8× gap.

Two caveats apply. KLD here is measured at prefill positions, so it does not capture drift that compounds over long generations. Absolute values also depend on the corpus and reference, which is why Unsloth's run and smcleod's run of the same UD-MLX-4bit file differ (0.0227 vs 0.0592). Compare rankings within one test, not raw numbers across tests.

How do oQ, standard Q, MXFP4/NVFP4, and UD MLX actually differ at the same bpw?

All four formats compress BF16 weights. They differ in where the extra bits go.

Plain Q is mlx-lm's round-to-nearest: fast, universal, and blind to which tensors are sensitive.

oQ measures each layer's sensitivity on a calibration set and spends the bit budget where the data says it matters. Per the oQ documentation, the base is affine group-64 at every level except 8-bit, which uses mxfp8 with group 32. The same doc reports large wins over mlx-lm at very low bit widths on Qwen3.5-35B-A3B: 64.0% vs 14.0% MMLU at 2-bit, and 85.0% vs 76.3% at 3-bit. At 4-bit the gap narrows to 83.3% vs 79.7%.

MXFP4 is the OCP microscaling format: 4-bit floats with one shared power-of-two exponent per 32 weights. FP elements handle outliers differently from integer grids, but a single E8M0 scale is coarse. NVFP4 uses the same 4-bit elements with a finer, 16-weight E4M3 scale, which is why it measures better in Unsloth's MLX table.

UD MLX carries Unsloth's GGUF-era Dynamic idea into MLX. On Qwen3.6-27B, smcleod.net's tensor audit found 258 overrides: 66 tensors at 8-bit, 3 at 5-bit and 189 at 4-bit. The 8-bit boosts fall on roughly every fourth attention layer, which puts the file at 8.6 effective bpw.

Which quantization runs on which hardware?

The measured rows come from the llama.cpp gpt-oss guide. There, gpt-oss-20b in native MXFP4 posts 221.95 tok/s (tg128) on an RTX 4090 and 161.77 tok/s on an RTX 3090. That contradicts the older advice that MXFP4 is a "trap" off Blackwell: the format decodes fine on Ada and Ampere. What those cards lack is native FP4 tensor-core math for prefill. A dash means no public measurement was found, not that the format is unsupported.

AMD Radeon cards run the same GGUF K-quant and UD files through ROCm or Vulkan. This revision found no cited public KLD or tok/s figures for MXFP4 or NVFP4 on consumer RDNA hardware, so it makes no claim either way. The earlier claims of "native MXFP6" on RDNA4 have been removed.

KLD benchmark: Qwen 3.6-27B and 35B-A3B across the MLX formats

These are the public numbers. The dense 27B test used a BF16 reference, 65,536 WikiText-2 tokens, and an Apple M5 Max with 128 GB (smcleod.net). Lower is better.

Qwen3.6-27B (dense), KLD vs BF16:

Qwen3.6-35B-A3B (MoE), KLD vs the 8-bit reference:

Qwen3.6-27B, Unsloth's own MLX run (MXFP4 and NVFP4 included):

Source: Unsloth Qwen3.6 docs.

Reading across the three tables gives four practical conclusions. At about 6 bpw on a dense model, plain Q and oQ6 are effectively tied, so pick the simpler recipe. At 4 bits, oQ4 and plain Q are within 3% on the dense model, but on the MoE oQ4 beats plain Q by about 46% and loses to DWQ. Within one quantizer's MLX lineup, NVFP4 ranks above MXFP4. UD-MLX-4bit costs 6-bit-class memory without 6-bit-class quality. smcleod's rule of thumb is to target the 5.8-7 bpw band from a well-made recipe, which "should be indistinguishable from bf16 in practice."

Prefill vs generation tok/s impact per format

Decode speed on a single stream is mostly limited by memory bandwidth: every token reads the active weights once. So a format's speed is largely its size. Each row below cites a measured figure. The ceiling column is bandwidth divided by the bytes read per token, using the Hugging Face weight-file size (for MoE, only the active share of that size). No row exceeds its ceiling.

Bandwidths are from Apple's M5 Pro/M5 Max announcement (M5 Max up to 614 GB/s) and NVIDIA's published board specs. Active-parameter figures for gpt-oss-20b are from OpenAI's model card (21B total, 3.6B active).

The first two rows are the useful comparison: the same model on the same chip in two formats. The UD-MLX build is 63% larger and decodes 37% slower, about what the size ratio predicts. Combined with its worse mean KLD in smcleod's test, it loses on both axes to plain 6-bit. The gpt-oss rows show MXFP4 working well as a native format on both Apple and Ada hardware. The Qwen3.8-27B row is plain llama-bench with no speculative decoding. The same thread reports much higher end-to-end figures with MTP or DFlash drafting, but those are speculative-decoding speedups, not format speed.

When do MXFP8 and NVFP4 beat integer quants on quality?

Earlier versions of this guide argued for "MXFP6 beats Q6_K." MXFP6 is defined in the OCP spec, but it is not a mode in MLX's quantize API. llama.cpp's FP4 GGUF types are MXFP4 and NVFP4. With no public KLD numbers for MXFP6 on local models, that section has been removed. The needle-in-a-haystack percentages it quoted could not be sourced.

What the public data does support:

• FP4 is a size/speed tool, not a quality upgrade over good integer quants. In Unsloth's GGUF KLD study, MXFP4_MOE (18.17 GB, mean KLD 0.0272) sits behind Unsloth's own Q4_K_M (18.49 GB, 0.0192) and UD-Q4_K_XL (19.17 GB, 0.0137). Unsloth notes that MXFP4 costs 4.25 bits per weight against Q4_K's 4.5, but still recommends Q4_K when choosing between them. On a model whose experts are released in MXFP4 the gap closes: AtomicChat's DeepSeek-V4-Flash ladder puts MXFP4_MOE (0.156042) and Q4_K_M (0.155595) within its ±0.003 noise band, and the Q4_K_M file is about 11% larger (171.8 GB vs 155.0 GB).
• NVFP4 beats MXFP4. The finer 16-weight E4M3 scale measured better in Unsloth's MLX run: a mean KLD of 0.0325 vs 0.0479 on Qwen3.6-27B.
• mxfp8 is how oQ8 is built. Per the oQ docs, oQ uses mxfp8 with group 32 at its 8-bit level. At 8 bits, every format sits close enough to BF16 that the choice rarely matters.
• The Blackwell payoff for NVFP4 is prefill and batch throughput, not single-stream quality. Unsloth reports its W4A4 NVFP4 Qwen3.6-27B at 125.9 decode tok/s on a B200 in vLLM, against 115.6 for NVIDIA's W4A16 build, with far higher batched throughput (Unsloth). The quality price of the A4 half is measurable: on Qwen3.8-27B, kelnei's NVFP4 card reports a mean KL of 0.0134 weight-only and 0.0259 with W4A4 emulated.

Verdict matrix

Bottom line

For late 2026, choose on memory first, then the recipe, then the format name. On a Mac with room for it, plain 6-bit or oQ6 of a dense model sits in the band where quality loss should be imperceptible. On MoE models, choose oQ6 at 6-bit and DWQ over oQ4 at 4-bit. Treat UD-MLX as experimental until Unsloth's MLX recipe matures. Its current Qwen3.6-27B 4-bit build is larger and slower than plain 6-bit and scores worse on KLD. On NVIDIA, UD-Q4_K_XL and the K-quants remain the GGUF quality leaders at 4-5 bpw. Use MXFP4 when a model ships natively in it and NVFP4 when you are on Blackwell and need throughput. Benchmark on KLD, not perplexity, and compare rankings within a single test.

A simple recommendation flowchart:

1. Are you on Apple Silicon with enough memory for about 6 bpw? → plain 6-bit or oQ6 (MoE: oQ6).
2. Are you on Apple Silicon and memory-limited to about 4 bpw? → dense: plain 4-bit or oQ4 (within 3% of each other); MoE: DWQ 4-bit.
3. Are you on Apple Silicon and need 2-3 bit? → oQ (the largest measured gains over plain Q).
4. Are you on an RTX 5090 or other Blackwell card? → UD-Q4_K_XL / K-quants for quality; NVFP4 (llama.cpp with PR #22196, or vLLM) for throughput.
5. Are you on RTX 4090 / 3090? → UD-Q4_K_XL or Q5 K-quants. MXFP4 is fine for models that ship in it, such as gpt-oss.
6. Are you on AMD, CPU or ARM? → GGUF K-quants / UD GGUF. There is no public data to justify FP4 here yet.

Related guides

• Best GPU for Local LLMs in 2026
• Ling-2.6-1T Local Inference Hardware
• Apple M5 Max for AI Workloads
• RTX 5090 Local Inference Deep Dive

Citations and sources

• smcleod.net: Measuring Model Quantisation Quality with KL Divergence (April 2026). Qwen3.6-27B and 35B-A3B MLX KLD tables; M5 Max; 65,536 WikiText-2 tokens.
• Unsloth: Qwen3.5 GGUF Benchmarks. GGUF KLD table including MXFP4_MOE vs Q4_K_M vs UD-Q4_K_XL; MXFP4 retirement note.
• Unsloth: Qwen3.6 How to Run Locally. MLX Dynamic KLD table (4-bit, NVFP4, MXFP4); NVFP4 vLLM decode figures.
• MLX documentation: mlx.core.quantize. Quantization modes affine, mxfp4, mxfp8 and nvfp4, and their group sizes.
• oMLX: oQ Quantization documentation. oQ design and the mlx-lm vs oQ accuracy table.
• llama.cpp PR #15091: gpt-oss support with native MXFP4. Added the MXFP4 data type to ggml (merged August 2025).
• llama.cpp PR #20644: NVFP4 dp4a kernel. Merged March 2026.
• llama.cpp PR #21074: generic NVFP4 MMQ kernel. Merged April 2026.
• llama.cpp PR #22196: Blackwell-native NVFP4. Merged April 28, 2026.
• llama.cpp Discussion #15396: running gpt-oss with llama.cpp. gpt-oss-20b MXFP4 llama-bench results on RTX 4090 and RTX 3090.
• oMLX benchmark: Qwen3.6-27B 4-bit on M5 Max. 32.1 tok/s decode at 4k context.
• oMLX benchmark: Qwen3.6-27B-UD on M5 Max. 20.2 tok/s decode at 4k context.
• oMLX benchmark: gpt-oss-20b-MXFP4-Q4 on M5 Max. 137.0 tok/s decode at 4k context.
• Hugging Face discussion: Qwen3.8-27B on RTX 5090. llama-bench UD-Q4_K_XL, tg128 76.0 tok/s.
• Apple Newsroom: M5 Pro and M5 Max (March 2026). M5 Max memory bandwidth up to 614 GB/s.
• OpenAI gpt-oss-20b model card. 21B total and 3.6B active parameters.
• akopytko/Qwen3.8-27B-NVFP4-GGUF model card (Hugging Face, README updated 2026-10-04) — Qwen3.8-27B mean KLD vs BF16 on wikitext (Unsloth NVFP4 0.070824, NVIDIA NVFP4+Q6_K 0.039314, Unsloth Q4_0 0.028317, Unsloth Q6_K 0.002857) and RTX 5090 llama-bench pp2048/tg128 throughput.
• kelnei/Qwen3.8-27B-NVFP4 model card (Hugging Face, August 2026). Mean KL vs BF16 0.0134 weight-only and 0.0259 W4A4-emulated; top-1 agreement 96.1% / 94.1%; vLLM 0.27.1 throughput on RTX PRO 6000 Blackwell.
• AtomicChat/dsv4-eval-artifacts dataset (Hugging Face, August 2026). DeepSeek-V4-Flash-0731 KLD ladder against a shared BF16 reference on wikitext-2 at ctx 5632, 8× RTX 5090; MXFP4 0.156403, MXFP4_MOE 0.156042, Q4_K_M 0.155595, UD-Q4_K_XL 0.155725.

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.

— SpecPicks Editorial