China Mobile's JT-4.1 Flash 236B is a sparse mixture-of-experts (MoE) release with 236 billion total parameters but only ~21 billion activated per forward pass — the "A21B" in the name. Per Artificial Analysis, it scores 39 on the Intelligence Index at roughly 14,000 output tokens per Index item and is classified as non-reasoning. Practically hosting it locally means the full 236B weights on disk, ~21B active at inference, and enough VRAM plus system RAM to hold either a heavily-quantized full model or a distilled/routed subset.
The rise of sparse MoE and why active-param count matters more than total
Sparse mixture-of-experts is the architecture that finally lets you say "236B parameter model" without needing datacenter-class hardware to run it. In a dense model, every token touches every parameter — a 70B dense model at fp16 wants ~140 GB of memory for a single forward pass. In a sparse MoE, a router picks two or three experts out of many per token, so only ~21B of the 236B ever run per forward step. Total-parameter count still sets the storage bill; only active-param count sets the per-token compute bill.
That gap is why JT-4.1 Flash 236B is the interesting case study for 2026 local rigs. The llama.cpp community has spent the last year turning MoE from "specialist datacenter architecture" into "you can quantize this and load it on a workstation", and every new open MoE release stress-tests those recipes. JT-4.1 Flash is the current test case: a non-reasoning model with 39 Intelligence Index performance, token-efficient at inference time (~14k tokens per Index run puts it inside the reasoning tier's efficiency envelope), and open enough that the community is already publishing quantized artifacts. If you understand what it takes to run this one, you understand the local-hardware envelope for MoE broadly.
Key takeaways
- 236B total / A21B active — routing gives you dense-70B-like compute cost per token.
- Intelligence Index 39 — competitive with mid-tier 32B–70B dense models on aggregate quality.
- ~14k output tokens per Index run — token-efficient (non-reasoning) despite competitive scores.
- Non-reasoning — answers directly, no visible chain-of-thought spend.
- Local floor for full-weight q4: 24 GB VRAM alone is not enough; you need VRAM + fast system RAM offload or a two-card build.
Why does A21B active change the local-hardware math?
Two numbers control what a 12 GB card can and cannot do with an MoE:
- Total quantized weight size on disk / in memory. This is what needs to be loadable somewhere — VRAM, system RAM, or streamed from NVMe. For a 236B model at q4, weights are ~118 GB.
- Active weight footprint per forward pass. This is what the GPU actually touches per token. For A21B at q4, that's ~10.5 GB — roughly what a 12 GB card can hold with room for KV cache.
The catch is the router. On every token, the model picks a different subset of experts. If the experts your prompt "routes to" all happen to live in system RAM rather than VRAM, you pay a PCIe-transfer cost per token, and throughput collapses. Full-weight local hosting of a 236B MoE therefore has three viable shapes: (a) keep all weights in fast VRAM (impossible on a single consumer card), (b) keep all weights in fast system RAM and stream to the GPU (works but slow), or (c) split weights across two cards with tensor parallelism (fastest, most expensive).
How much VRAM does a 236B MoE really need at q4?
Rough memory ledger for common quants of a 236B MoE, weights only, before KV cache:
| Quant | Full-weight size | Active-weight footprint | Notes |
|---|---|---|---|
| q2_K | ~64 GB | ~5.8 GB | Meaningful quality loss on MoE |
| q3_K_M | ~78 GB | ~7 GB | Aggressive but usable |
| q4_K_M | ~118 GB | ~10.5 GB | Community-recommended sweet spot |
| q5_K_M | ~140 GB | ~12.6 GB | Overflows 12 GB VRAM even active-only |
| q8_0 | ~230 GB | ~21 GB | Requires ≥24 GB VRAM or dual cards |
| fp16 | ~470 GB | ~42 GB | Datacenter territory |
Compare that against the TechPowerUp RTX 3060 specs — 12 GB VRAM, 360 GB/s bandwidth. Active-weight q4_K_M fits, but the rest of the weight file lives elsewhere, and system RAM bandwidth becomes the ceiling.
Benchmark table
| Model | Intelligence Index | Tokens per Index run | Class |
|---|---|---|---|
| JT-4.1 Flash 236B A21B | 39 | ~14k | Non-reasoning MoE |
| Qwen 3 32B (dense) | ~44 | varies | Dense reasoning-capable |
| Llama 3.1 70B (dense) | ~40 | varies | Dense |
| DeepSeek V3 MoE (671B/A37B) | ~55 | ~20k | Non-reasoning MoE |
JT-4.1 Flash trades some peak quality for materially lower active-param cost. Per Artificial Analysis's JT-4.1 Flash page, the token efficiency is the actual selling point: you get non-reasoning-tier response speed at Index-competitive scores.
Quantization matrix — VRAM, tok/s, quality on MoE weights
Approximate throughput on a single RTX 3060 12 GB paired with a Ryzen 7 5800X, 64 GB dual-channel DDR4-3600, weights streamed from a Samsung 970 EVO Plus NVMe:
| Quant | Weights fit? | Throughput | Quality loss |
|---|---|---|---|
| q2_K | 64 GB — RAM only | 4–6 tok/s | Meaningful |
| q3_K_M | 78 GB — RAM only | 3–5 tok/s | Noticeable |
| q4_K_M | 118 GB — RAM+NVMe swap | 1–2 tok/s | Recommended if fits |
| q5_K_M | 140 GB — NVMe swap | <1 tok/s | Unusable single-card |
| q8_0 | 230 GB — NVMe swap | <0.3 tok/s | Datacenter-tier |
The RAM-only rows require 64 GB or 128 GB of system memory. NVMe-swap rows require a fast SATA fallback drive is not enough — you need NVMe with high sustained random reads.
The realistic budget path
For a single 12 GB card, the honest recommendation is: don't try to run the full 236B model. Run a distilled or expert-pruned variant, or run a dense 30B–32B q4 that fits in VRAM, and treat JT-4.1 Flash as a hosted API for when you specifically want its output. If you must self-host 236B, the workable configurations are:
- Budget floor: RTX 3060 12 GB + 64 GB DDR4 + fast NVMe → q2_K, RAM-only, ~5 tok/s. Playable for background jobs, painful for interactive chat.
- Recommended: 24 GB card (RTX 3090/4090 used) + 64 GB DDR4 → q3_K_M active-in-VRAM, RAM-resident inactive weights, 10–15 tok/s.
- Comfortable: 2× 24 GB cards + 128 GB DDR4 → q4_K_M with tensor parallelism, 25–40 tok/s.
Prefill scales with router hit rate and PCIe topology more than raw compute; a single 12 GB card is prefill-bound on any prompt longer than a few thousand tokens.
Prefill vs generation and context-length impact for MoE routing
MoE prefill is dominated by two costs: the FFN pass on active experts (compute-bound), and the router lookup (memory-bound). On a single 12 GB card with weights in RAM, prefill is roughly 1/3 the throughput of a dense equivalent because the router repeatedly pulls "cold" experts from RAM. Generation is smoother once the model warms — the working set of active experts tends to stabilize per prompt.
Context length is the other consideration. KV cache for a 236B MoE at q4 with 8k context runs ~3 GB. At 32k that jumps to ~12 GB — half the total VRAM of a 24 GB card. Long-context work on a single 12 GB consumer card is not viable at any quant; 8k–16k is the honest envelope.
Perf-per-dollar: entry 12 GB rig vs 24 GB step-up vs cloud
- Entry 12 GB rig (RTX 3060 + 5800X + 64 GB DDR4 + NVMe) — around $1,000–1,200 in 2026. Runs 236B q2/q3 slowly, runs dense 30B q4 comfortably. Best for a builder who mostly runs dense 30B and occasionally wants to touch a big MoE.
- 24 GB step-up (used RTX 3090 + 5800X + 128 GB DDR4) — around $1,500–1,900. Runs 236B q3/q4 at interactive speeds, no cloud round-trip.
- Hosted API — pennies per Index run at JT-4.1 Flash's token-efficiency. Break-even against a $1,200 local build is at very high monthly usage or when you need offline/privacy guarantees.
For most solo builders, hosted API beats local for JT-4.1 Flash specifically. Local wins when you also run other models on the same rig (Qwen 3, Llama 3.x, Stable Diffusion, ComfyUI, dev workflows).
Common pitfalls
- Buying 32 GB of system RAM. 236B q4 will not fit at all. 64 GB is the floor; 128 GB removes NVMe-swap thrash.
- Slow NVMe. MoE routing thrashes storage on cold-cache prompts; a 3,500 MB/s NVMe is fine, a SATA SSD is not.
- Wide-context prompts on a 12 GB card. KV cache eats your active-weight budget before generation ever starts.
- Assuming "big total params → big quality". JT-4.1 Flash is A21B; treat it as a 21B-class model for compute, a 32B-class model for quality.
Bottom line: who should attempt a local JT-4.1 Flash deployment
A local JT-4.1 Flash deployment on the MSI RTX 3060 12 GB makes sense if: you already own the hardware, you enjoy the exercise of tuning MoE routing, you want to compare open MoE quality without a bill, and you accept 1–5 tok/s. It does not make sense as your only path — hosted API is faster and cheaper for daily use. A 24 GB card and 128 GB of DDR4 (paired with a proven Ryzen 7 5800X foundation and a fast Samsung 970 EVO Plus NVMe for weights, with a Crucial BX500 SATA SSD as a scratch drive) is the realistic self-host build for this model class.
Related guides
- vLLM vs llama.cpp on a 12 GB GPU
- Best NVMe SSD for local LLM model storage
- Best GPU for Llama 70B local
- Dual-channel RAM and local LLM inference
- Ryzen AI Halo vs DGX Spark local AI mini-box
Citations and sources
- Artificial Analysis — JT-4.1 Flash 236B A21B model page
- TechPowerUp — GeForce RTX 3060 specifications
- llama.cpp GitHub — MoE quantization support
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
FAQ
What is JT-4.1 Flash 236B and who released it? JT-4.1 Flash 236B is a sparse MoE model published by China Mobile with 236B total parameters and roughly 21B active per forward pass. Per Artificial Analysis it scores 39 on the Intelligence Index and uses about 14,000 output tokens per Index run, placing it as a non-reasoning model that is nonetheless token-efficient enough to sit near the reasoning tier's cost envelope.
Does "236B" mean I need datacenter hardware to run it? Not quite. The total parameter count sets the memory-footprint bill (weights on disk plus loaded memory), but only the ~21B active portion runs per token, so per-token compute cost is closer to a dense 21B model. What you cannot skip is enough total memory — VRAM plus system RAM — to hold at least a quantized copy of the full weight file. On a single 12 GB consumer card the honest answer is: yes, mostly.
How much VRAM and system RAM is realistic? For a heavily quantized q2 or q3 build you need at least 64 GB of system RAM plus a 12 GB GPU and a fast NVMe. For a comfortable q4 with interactive speeds, plan for a 24 GB GPU (used RTX 3090/4090), 128 GB of DDR4, and NVMe storage. Anything below those budgets works only as a slow batch server, not as a chat backend.
Is JT-4.1 Flash 236B open-weight? Community distributions and llama.cpp quant recipes exist as of coverage. Read the specific model card and license on Artificial Analysis and the source repo before deploying; MoE releases from various labs have varying license terms and some are gated behind acceptance forms.
Should I run this locally or use a hosted API? For most solo builders, hosted API beats local for JT-4.1 Flash specifically because token-efficiency makes hosted usage cheap and the local hardware envelope is punishing. Local becomes attractive when you also run other models on the same rig, when privacy or offline is required, or when your monthly usage is high enough to amortize a $1,500+ workstation.
