Key Takeaways
- Iris-mini fits a 12 GB card only with expert offload. Even its Q3_K_M file is 17.17 GB, per the mradermacher GGUF repo.
- Iris-pro does not fit at all. It has 397B parameters and 806.8 GB of BF16 weights, per its Hugging Face card.
- The KV cache is small. Only 10 of Iris-mini's 40 layers use full attention, so the full 256K window needs about 2.5 GiB of FP16 cache (computed from config.json).
- The base model's speed on a 3060 is ~38 tok/s. That was measured with
-ncmoe 24and 9.8 GB of VRAM in use (InsiderLLM). - The quality gap is 6.4 BrowseComp points. Iris-pro scores 88.6 and Iris-mini scores 82.2 (arXiv HTML, Table 1).
- Context management is worth up to 21.2 points to Iris-mini. Its BrowseComp score goes from 64.7 without it to 85.9 with discard-all plus retry, per the Iris README.
Step 0: what a "search agent" actually demands from your hardware
A chat model answers once. A search agent loops. The Iris-Harness README describes the loop as "a single-agent ReAct loop with two tools — web_search and scrape_website". The reference configuration allows up to 600 turns. Every turn appends tool output to the context, and the model re-reads that growing context before its next decision.
That shape puts three separate loads on the hardware:
- Weights that stay resident. An agent session can last many minutes, so the model has to sit in memory the whole time. Loading it again for each query is not an option.
- A context that keeps growing. The headline Iris results reset context at 131,072 tokens, according to the harness configuration in the Iris repository. On most dense models, a context of that size alone would fill a 12 GB card with KV cache.
- Prefill repeated every turn. Each new search result has to be processed before generation continues. Prefill speed therefore sets the wall clock as much as generation speed does.
There is also a component most local guides leave out. The harness's scrape_website tool forces LLM extraction of each page it fetches, and the harness docs call the summary model "required, not optional". A faithful local setup therefore runs two models, or it pays an API for page summaries. Keep that in mind when you budget VRAM.
Parameter count is not the constraint to start with. The useful questions are how much of the model has to live in fast memory, and how quickly the rig can re-read a 50K-token history.
Spec delta table
| Model | Parameters (total / active) | Q4_K_M weights size | KV cache at 32K (FP16) | Fits a 12 GB card? |
|---|---|---|---|---|
| Iris-mini | 35B / 3B | 21.71 GB | ~0.31 GiB | With expert offload to system RAM |
| Iris-pro | 397B / 17B | No GGUF published (BF16 is 806.8 GB) | ~0.46 GiB | No |
The parameter counts and base models are listed on the Iris-mini and Iris-pro model cards. The Q4_K_M size comes from the mradermacher GGUF repo.
The KV figures are arithmetic from each config.json. Iris-mini has 10 full-attention layers, 2 KV heads and a head dimension of 256. That gives 2 × 10 × 2 × 256 × 2 bytes = 10,240 bytes per token at FP16. Iris-pro has 15 full-attention layers, so it needs 15,360 bytes per token. The remaining layers use Gated DeltaNet linear attention, which keeps a fixed-size state instead of a per-token cache. That design is why both models tolerate long contexts so cheaply.
The KV cache is not what keeps Iris-pro out of reach. The weights are. Even a hypothetical 2-bit quant of a 397B model would need roughly 100-130 GB, which is ten times the card's memory before any cache.
Benchmark table: reported task success for Iris-mini vs Iris-pro
All scores below are pass@1 with an LLM judge. They come from Table 1 of the Iris paper, and they match the GitHub README. Headline rows use the discard-all context-management setting.
| Model | BrowseComp | BrowseComp-ZH | DeepSearchQA (F1) | HLE (text) |
|---|---|---|---|---|
| Iris-mini (context management on) | 82.2 | 84.8 | 86.9 | 52.3 |
| Iris-mini (no context management) | 64.7 | 72.3 | 81.0 | 43.2 |
| XYZ-Aquila-mini (reported) | 78.8 | 82.9 | 89.5 | 51.1 |
| Iris-pro (context management on) | 88.6 | 85.1 | 92.9 | 56.4 |
| Iris-pro (no context management) | 72.6 | 76.8 | 86.4 | 50.8 |
| XYZ-Aquila-pro (reported) | 84.8 | 85.1 | 92.5 | 53.3 |
Two things stand out. First, the gap between the two Iris models is small on some benchmarks, only 0.3 points on BrowseComp-ZH, and larger on others: 6.4 on BrowseComp and 6.0 on DeepSearchQA. Second, the harness matters more than the model upgrade. Turning on context management adds 17.5 BrowseComp points to Iris-mini. Moving from Iris-mini to Iris-pro adds 6.4. The Decoder reports the same effect: the smaller model "needs more steps for the same tasks and hits the context limit more often."
The paper does not report latency, and neither does the README. Any wall-clock comparison has to come from hardware measurements of the base architecture, covered below.
Quantization matrix for both models
Iris-mini sizes come from the mradermacher/Iris-mini-GGUF file listing. The throughput bands are measurements of the base model Qwen3.6-35B-A3B on an RTX 3060, cited in the next section. Treat them as an inference for Iris-mini, not as a measurement of it.
| Quant | Iris-mini file size | VRAM needed for full GPU residency | Throughput band on one 3060 (base model) | Quality loss |
|---|---|---|---|---|
| Q3_K_M | 17.17 GB | ~18 GB+ | ~20-23 tok/s at IQ3_XXS class with offload | Not published for Iris-mini |
| Q4_K_M | 21.71 GB | ~23 GB+ | ~38 tok/s with -ncmoe 24 | Not published for Iris-mini |
| Q5_K_M | 25.35 GB | ~27 GB+ | Offload-bound; no public 3060 figure | Not published for Iris-mini |
| Q6_K | 29.21 GB | ~31 GB+ | Offload-bound; no public 3060 figure | Not published for Iris-mini |
| Q8_0 | 37.80 GB | ~39 GB+ | Offload-bound; no public 3060 figure | Not published for Iris-mini |
The "VRAM needed" column is the file size plus a small allowance for cache and compute buffers. No row fits entirely in 12 GB. AllSpark has not published a quantization-quality evaluation for Iris-mini, so the quality column states that honestly rather than borrowing a generic perplexity figure. Agent success rates are more sensitive to quantization than chat quality, so measure your own task success before settling on a low quant.
For Iris-pro, no GGUF appeared on Hugging Face at the time of writing. Its reference serving command on the model cards uses SGLang with --tp-size 4, meaning tensor parallelism across four datacenter GPUs. Every Iris-pro row in a 12 GB matrix would read "does not fit".
Running Iris-mini on one 12 GB card
The two street-available 12 GB cards SpecPicks tracks are the MSI Gaming GeForce RTX 3060 12GB and the ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB. Both use the same GPU. The NVIDIA RTX 3060 family page lists 12 GB of GDDR6 on a 192-bit memory interface and 170 W of graphics card power. Measured results for the chip are collected on the RTX 3060 benchmark page.
The technique that makes a 21.71 GB model usable on a 12 GB card is expert offload. In a mixture-of-experts model, most parameters sit in expert feed-forward blocks, and only 8 of Iris-mini's 256 experts fire per token, per its config.json. llama.cpp's --n-cpu-moe N (-ncmoe) keeps attention, the shared expert and routing on the GPU, and moves the expert weights of the first N layers to system RAM.
The most detailed public 3060 sweep, from InsiderLLM, ran the base model at unsloth UD-Q4_K_M on an i7-7700 with 32 GB of DDR4-2133:
-ncmoe setting | Generation (tok/s) | VRAM used |
|---|---|---|
| 40 (all experts in RAM) | 28.1 | 2.5 GB |
| 32 | 31.8 | 6.1 GB |
| 24 | 38.9 | 9.8 GB |
| 20 | 42.6 (out of memory once context grew) | 11.7 GB |
At -ncmoe 24, the same source reports that generation "holds ~38 tok/s, flat through 8K of context": 38.9 empty, 38.5 at 4K and 38.2 at 8K. Prompt processing ran at 413, 394 and 386 tok/s at those depths. The page adds a warning for cards that also drive a monitor: the display costs "~0.5-1 GB", enough to push -ncmoe 24 into an out-of-memory error, so step up to 26 or 28.
A second write-up, by Jean Brito, used the smaller UD-IQ3_XXS file (13.2 GB) on an older Xeon E5-2650 v4, with q8_0 KV cache and a 65,536-token context. It measured 22.9, 21.1 and 19.7 tok/s at -ncmoe 25, 30 and 35, and "OOM immediately" without the flag. The slower host CPU and memory explain most of the difference from the i7 run.
Practical settings for Iris-mini, based on those two write-ups:
- Start at Q4_K_M with
-ngl 99 -ncmoe 26 -fa on -np 1, then lower-ncmoeuntil VRAM use sits around 10.5 GB. - Set
-np 1. Each extra parallel slot allocates its own cache. - Budget 32 GB of system RAM. With about 10 GB of weights on the card, roughly 12 GB of the Q4_K_M file lives in RAM, and the OS, harness and browser tools need the rest.
Running Iris-pro: what has to give
For Iris-pro, the usual levers don't add up. Quantization cannot close a 67× gap between 806.8 GB of weights and a 12 GB card. KV-cache quantization saves megabytes on a problem measured in hundreds of gigabytes. CPU offload to an AMD Ryzen 7 5800X runs into two walls. First, even an aggressive low-bit Iris-pro quant would need roughly 100-130 GB of memory, which fills or exceeds the largest DIMM kits an AM4 board takes. Second, dual-channel DDR4-3200 moves about 51.2 GB/s (2 × 25.6 GB/s), so every decoded token would be slow even though only 17B parameters are active. At 17B active parameters and ~2.5 bits per weight, each token reads about 5 GB, which caps generation near 10 tok/s before any compute overhead. And no Iris-pro GGUF exists to test that ceiling.
What actually has to give is the hardware class. Iris-pro is sized for the reference four-GPU SGLang deployment on its model card, or for a hosted endpoint. If your use case really needs Iris-pro's extra 6.4 BrowseComp points, the honest local answer is "not on a 3060, and not on two of them."
The realistic substitute is Iris-mini with context management turned on. Per the paper, it scores 82.2 on BrowseComp, above Iris-pro's no-context-management score of 72.6. On this benchmark, a well-configured small agent beats a poorly configured large one.
Prefill vs generation: why a search agent's re-prefill loop dominates the wall clock
Generation speed is the number people quote. Prefill speed is the one an agent feels. Take a turn that pulls in a 6,000-token scraped page. At the measured ~390 tok/s prompt-processing rate on a 3060 (InsiderLLM), that page takes about 15 seconds to ingest. Generating a 300-token plan at ~38 tok/s then takes about 8 seconds. Over a 40-turn research task, the reading costs twice as much time as the writing.
Two mitigations matter more than hardware:
- Prompt caching. llama.cpp reuses the cached prefix when a request extends the previous one, so each turn only prefills the new tool output rather than the whole history. Harnesses that rewrite earlier messages break that reuse.
- Context discard. The Iris harness's discard setting clears accumulated tool output at a threshold. That lowers the prefill cost of later turns, and per the paper it also raises accuracy.
Iris-mini suffers more from the re-prefill loop than Iris-pro would. The Decoder notes that it "needs more steps for the same tasks", and more steps means more prefill passes. On a 3060, expert offload also hits prefill harder than generation, because every prompt token routes through experts held in system RAM.
Context-length impact: 8K / 32K / 64K budgets on 12 GB
Because only 10 of 40 layers keep a KV cache, Iris-mini's context cost is unusually low. Computed from config.json at 10,240 bytes per token for FP16:
| Context | KV cache FP16 | KV cache q8_0 | Share of a 12 GB card (FP16) |
|---|---|---|---|
| 8K | ~78 MiB | ~41 MiB | <1% |
| 32K | ~313 MiB | ~166 MiB | ~2.5% |
| 64K | ~625 MiB | ~332 MiB | ~5% |
| 131K (harness discard point) | ~1.25 GiB | ~0.66 GiB | ~10% |
| 262K (maximum) | ~2.50 GiB | ~1.33 GiB | ~21% |
The linear-attention layers add a fixed recurrent state per sequence, a few tens of megabytes, that does not grow with context. For comparison, a dense 14B model with full attention in every layer needs several gigabytes of cache at 64K. That is why dense models of similar quality stall on 12 GB cards at long context, while this architecture keeps going.
The practical effect is that on a 3060, context is not the constraint for Iris-mini; expert placement is. Adding context costs a few hundred megabytes, and freeing those megabytes costs about one extra -ncmoe layer. If VRAM is tight, q8_0 cache (--cache-type-k q8_0 --cache-type-v q8_0, which requires flash attention) halves the cache with little risk. Brito's 65K-context run used exactly that.
The cheap always-on host: AMD Ryzen 5 5600G plus a 1TB Crucial BX500
A search agent runs best as a box that stays on. The AMD Ryzen 5 5600G is a sensible host for that box. Its integrated Radeon graphics can drive the console, which leaves the 3060's full 12 GB for the model. That matters here, because the InsiderLLM measurements show a display costing 0.5-1 GB of VRAM. Its six cores are enough to feed a GPU-resident model, but offloaded experts run on the CPU, so a faster host improves offloaded generation.
For the model library, the Crucial BX500 1TB SATA SSD is rated "up to 540MB/s" in its product listing. At that rate, the 21.71 GB Q4_K_M file loads in about 40 seconds (21,710 MB ÷ 540 MB/s), compared with a few seconds on a fast NVMe drive. For an always-on agent that loads once and runs for days, 40 seconds does not matter. A 1 TB drive holds Iris-mini at two quants, a summary model and a spare model with room to spare.
Perf-per-dollar and perf-per-watt for a 24/7 search agent
Catalog prices on SpecPicks at the time of writing are $479.99 for the MSI RTX 3060 12GB, $199.99 for the Ryzen 5 5600G and $163.49 for the BX500 1TB. Prices change often, so check the linked product pages for current figures.
| Configuration | Can run | Measured or estimated generation | Card power budget |
|---|---|---|---|
| One RTX 3060 12GB + 32 GB RAM | Iris-mini Q4_K_M with offload | ~38 tok/s (base model, InsiderLLM) | 170 W (NVIDIA) |
| One RTX 3060 12GB, slower host, IQ3 quant | Iris-mini IQ3-class | ~20-23 tok/s (Brito) | 170 W |
| Any consumer rig | Iris-pro | Does not fit | n/a |
Power use during an agent session is bursty. The card spikes during prefill and sits near idle while the agent waits on a search API. The 170 W figure is NVIDIA's graphics card power rating, not a measured average. An always-on box spends most of its day idle, so idle draw usually matters more to the power bill than peak draw.
On cost per BrowseComp point, Iris-mini on a single 3060 has no local competitor at this price. Iris-pro's extra 6.4 points require hardware that costs more than an order of magnitude more.
Common pitfalls
- Leaving
-npat its default. llama-server allocates parallel slots and each one gets its own cache. Set-np 1for a single-user agent. - Tuning
-ncmoeon an empty context. The InsiderLLM sweep shows-ncmoe 20reaching 42.6 tok/s, then running out of memory once real context arrived. Tune with a full-length prompt. - Skipping the summary model. The harness requires one for
scrape_website. Running it on the same card cuts into Iris-mini's VRAM, so consider a small CPU model or a cheap API. - Assuming tool-call parsing works in every server. The Iris cards document SGLang with the
qwen3_codertool-call parser. The Iris repository does not mention llama.cpp, so test tool calls end to end before trusting a long run.
Verdict matrix
Get Iris-mini if… you have one 12 GB card and 32 GB of system RAM, you want a private research agent, and you will run it with the harness's context management turned on.
Get Iris-pro if… you have (or rent) a multi-GPU datacenter node, or an endpoint that serves it, and the 6.4-point BrowseComp gain justifies that cost.
Stay on a hosted search API if… your query volume is low, you need answers that depend on fresh web data at scale, or you don't want to run two models plus a Serper key on a home box.
Recommended pick
Iris-mini is the only one of the two that belongs on an RTX 3060, and it is a strong fit there. Because 30 of its 40 layers use linear attention, the 256K context costs 2.5 GiB of cache rather than tens of gigabytes. Because only 3B parameters are active, expert offload keeps the base architecture near 38 tok/s on a measured 3060 build. Pair an MSI RTX 3060 12GB with 32 GB of RAM and a Ryzen 5 5600G host, and you have a private search agent that scores 82.2 on BrowseComp in its published configuration.
Bottom line
Iris-mini fits a 12 GB card through expert offload, and Iris-pro doesn't fit any consumer rig. On 12 GB, the benchmark gain from turning on context management (+17.5 BrowseComp points) is larger than the gain from upgrading models (+6.4). Set up the harness carefully before you consider spending more on hardware.
Related guides
- Which LLMs actually fit an RTX 3060 12GB in 2026
- RTX 3060 benchmarks
- The Q4_K_M agent trap: KV-cache quant math
- Best parts for a dual RTX 3060 24GB local LLM build
Live price comparison
See current pricing for the two 12 GB cards on the MSI Gaming GeForce RTX 3060 12GB and ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB product pages. For a CPU-offload host, compare the AMD Ryzen 7 5800X against the AMD Ryzen 5 5600G. Prices may vary.
Citations and sources
All sources accessed 2026-09-17.
- Iris: Climbing to the Search Frontier (arXiv 2609.04304): the paper introducing Iris-mini and Iris-pro.
- Iris paper, HTML version: Table 1 benchmark scores with and without context management.
- AllSpark-Research/Iris on GitHub: the README benchmark table, Iris-Harness tool loop, turn cap and required API keys.
- AllSpark-Research/Iris-mini on Hugging Face: model card, license, base model and SGLang serving command.
- Iris-mini config.json: layer count, KV heads, head dimension, attention layout and expert count used for the KV math.
- AllSpark-Research/Iris-pro on Hugging Face: parameter count and BF16 weight size.
- mradermacher/Iris-mini-GGUF: GGUF quant file sizes.
- The Decoder: Iris-mini and Iris-pro: release coverage and context-management effect.
- InsiderLLM: Qwen3.6-35B MoE locally: llama-bench
-ncmoesweep of the base model on an RTX 3060 12GB. - Jean Brito: Qwen3.6-35B MoE on an RTX 3060 with -ncmoe: IQ3_XXS runs at 65K context with q8_0 cache.
- NVIDIA GeForce RTX 3060 family page: 12 GB GDDR6, 192-bit memory interface, 170 W graphics card power.
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
