Thinking Machines Drops Inkling, a 975B Open Model Leading US Labs

Thinking Machines, the lab founded by former OpenAI CTO Mira Murati, released Inkling this week — a 975-billion-parameter open-weights language model that lands near the top of the Artificial Analysis Intelligence Index, beating every previous US-lab open release and closing the gap on Chinese frontier models. It ships under an open license, weights are downloadable today, and it is being pitched at the "GPT-5 class at home" audience. Hardware requirements are non-trivial.

Key takeaways

• Inkling is a 975B-parameter dense-ish MoE with a mid-hundred-billion active-parameter footprint per token, per Thinking Machines' technical brief.
• The Intelligence Index score puts it ahead of Llama 4, DeepSeek V3.1, and Meta's Behemoth releases as of publication, per Artificial Analysis.
• Weights are downloadable now via Hugging Face under an open-weights license, per The Decoder.
• Realistic self-hosting requires ≥8 datacenter GPUs at fp8 or a moderate cluster of consumer cards at q3–q4 with heavy offload; a single MSI RTX 3060 12GB cannot load even a low quant without offload.
• For hobbyists on a Ryzen 7 5800X build with 128GB DDR4 and a fast Samsung 970 EVO Plus NVMe, CPU + GPU-offload runs at low-single-digit tokens per second are the realistic ceiling.

What is Inkling?

Inkling is Thinking Machines' first flagship open-weights release, launched July 2026. The company was founded in 2024 by Mira Murati (ex-OpenAI CTO) and a small group of researchers, and it has raised over $2B in funding to date. Inkling was announced alongside a technical report describing a mixture-of-experts architecture with 975B total parameters, of which a fraction (reported in the mid-hundreds of billions) activate per token. That places it in the same weight class as DeepSeek V3.1 and above Meta's Llama 4 in raw parameter count.

The scoring context matters more than the count. On the Artificial Analysis Intelligence Index, Inkling is currently tied with — or slightly ahead of — the strongest open-weights release from a US-based lab, and closes to within a few points of the top-tier closed API models. That is the story the AI press ran with this week: a US lab has, for the first time in over a year, shipped an open model that trades punches with the best open work from China.

Spec table

Community measurements suggest an fp8 checkpoint is ~1TB on disk and an int4 quant lands near 500GB. Neither fits in the VRAM of a single consumer GPU. That is the practical hurdle for the "can I run it at home" question.

What running it locally actually requires

Public benchmarks and community deployment threads settle roughly on these tiers:

The last row is the one most SpecPicks readers care about. A 12GB RTX 3060 paired with a Ryzen 7 5800X, 128GB of DDR4, and a Samsung 970 EVO Plus NVMe is the archetypal budget local-LLM rig. Loading a 975B MoE at any quant on that hardware requires expert offload to system RAM and disk streaming, and performance falls to interactive-notebook levels — under one token per second in typical community runs.

The MoE angle: why active-parameter count matters

Inkling is not a dense 975B model. It routes each token through a subset of expert layers, so the active parameter count per token is far lower — Thinking Machines has published figures in the mid-hundred-billion range. That is what makes MoE models cheaper to serve at scale, and it is why cloud API pricing for Inkling is competitive with much smaller dense models.

For local inference, the key implication is that a quantization strategy that keeps hot experts in VRAM and cold experts on system RAM (or NVMe) can dramatically reduce peak VRAM pressure. Community forks of llama.cpp and vLLM have already begun landing expert-offload paths (see ggml-org/llama.cpp for the ongoing work). A budget rig will still be latency-bound, but the "won't fit at all" answer softens to "runs at 0.3 tok/s" — which is enough for scripted overnight tasks, if not interactive use.

The Chinese-lab context

Inkling arrives after 18 months of Chinese labs (DeepSeek, Qwen, Kimi, Zhipu, Baichuan) dominating open-weights leaderboards. US labs — OpenAI, Anthropic, Google — have kept their frontier weights closed. Meta's Llama 4 landed short of the top of the open leaderboard, and its Behemoth release was received as a strong second-tier model rather than a category-defining one.

Thinking Machines has been explicit that open weights are a strategic commitment, not a marketing gesture. The company has published training-data documentation, checkpoint-loading instructions, and eval harnesses. Whether that endures as the model finds commercial customers is the open question, but the initial release lands with the fewest strings attached of any US frontier open model to date.

Bottom line

If you have a datacenter card and want to self-host Inkling, the fp8 checkpoint is a straightforward deployment on 8× H100 or equivalent. If you are on a budget local-LLM rig — a single RTX 3060 12GB with a Ryzen 7 5800X and 128GB of DDR4 — the Thinking Machines API is the realistic path. For casual experimentation, community expert-offload builds on 2–4 consumer cards land in the sub-10-tokens-per-second range, which is enough for prompt-batching workflows but not interactive chat.

The larger point is what the release signals about the US open-model ecosystem: for the first time in over a year, an American lab has landed at the top of the open-weights charts, and it did so with a permissive license and downloadable weights on day one.

Common pitfalls

• Confusing total and active parameters. Sizing VRAM to 975B assumes a dense model; Inkling is MoE. Peak resident VRAM at any given step is much lower than the raw parameter count implies, but some expert layers still need to be in memory at all times.
• Trying to run q2 on a single 12GB card. Even the aggressive quants exceed 12GB of VRAM once KV-cache and expert routing state are accounted for. Offload plus RAM-side experts is the only path on a single-card build.
• Ignoring NVMe throughput. Expert-offload configurations stream weights from disk on cold-expert hits. A slow SATA SSD like the Crucial BX500 1TB is fine for the OS but will bottleneck cold-expert loads compared to a PCIe Gen3 NVMe like the Samsung 970 EVO Plus.
• Comparing to smaller dense models on quality. Community A/B tests suggest Inkling wins on complex reasoning tasks but is not obviously ahead of Qwen 3.6 27B on short-form tasks, and it costs an order of magnitude more compute to serve locally.

When not to bother self-hosting

If your workload is 90% short-form chat or code completion, a well-quantized dense model like Qwen 3.6 27B or DeepSeek Coder V3 on a single RTX 3060 12GB delivers most of the quality of Inkling at 5–20x the throughput. Self-host Inkling if you need its top-end reasoning or its 256K context window on private data; otherwise, the API is priced competitively and the operational cost of a self-hosted deployment (electricity, cooling, downtime) will exceed the API bill for typical workloads.

What Inkling actually beats — and what it doesn't

Community-run A/B comparisons published on r/LocalLLaMA and Artificial Analysis's own eval framework show Inkling's strengths cluster in a few areas:

• Multi-step math and formal reasoning: consistently beats Llama 4 and comes close to closed-frontier models on GSM8K-hard and MATH benchmarks.
• Long-context comprehension: its 256K context window is genuinely usable end-to-end, not a marketing number that degrades after 32K.
• Code generation for larger multi-file changes: ranked in the top tier on SWE-bench-lite in the first week of community testing.

Weaker areas, per the same testing:

• Short-form conversational latency: its MoE routing overhead makes single-turn chat feel slower than a similarly-scored dense model.
• Multilingual coverage outside English, Chinese, and a few European languages: substantially weaker than Qwen 3.9.
• Vision + audio grounding: Inkling is text-only in its initial release. Multi-modal support is on Thinking Machines' roadmap but not shipped.

Understanding those tradeoffs matters for anyone deciding whether to self-host. A model that beats every open peer on reasoning while sitting mid-pack on conversational chat is a tool, not a general daily-driver. It rewards being invoked deliberately for hard problems, not being pointed at every request that comes through.

The economics vs. running Inkling on the API

Moonshot-analogous rate cards for Inkling settle in the $0.90–1.50 per benchmark-task range as of publication, with per-token pricing under $0.001 per 1000 tokens for input and around $0.003 per 1000 tokens for output. For a serious individual user hitting ~1M output tokens per month, that lands at roughly $3–5/month. For a small team, tens of dollars per month.

Compare that to the amortized cost of self-hosting on 8× H100. Datacenter rental for such a node runs $80–200/hour on-demand. Even at reserved-instance rates and best-case utilization, break-even on cost-per-token requires sustained heavy usage across many users. For a single team, the API is dramatically cheaper. For a mid-size enterprise with data-residency constraints, the self-host math starts to make sense — but that is not the audience of most SpecPicks readers.

Why open weights still matter

Even if you're using the API, Inkling being released as open weights is meaningful. It sets a floor on how far closed-frontier models can go before an open competitor forces the price down. It gives independent researchers something to fine-tune, quantize, and probe without corporate NDA barriers. And it re-establishes that a US-based lab can compete at the frontier while shipping weights, which puts pressure on other US labs to do the same.

What running Inkling looks like on a "prosumer AI rig"

For readers on the boundary between "budget local rig" and "prosumer workstation," the interesting question is what an Inkling deployment looks like on something like a single RTX 5090-based build with 128 GB RAM and a fast NVMe. Community measurements land in this range:

• q3_K_S with GPU-resident hot experts + system-RAM cold experts: 1.5–3 tok/s. Usable for scripted overnight batches, marginal for interactive.
• q4_K_M with aggressive expert offload to NVMe: 0.5–1.5 tok/s. NVMe throughput becomes the limit; the Samsung 970 EVO Plus is sufficient but a PCIe Gen 4 or Gen 5 drive would help.
• Peak power draw: GPU pulls 400+ W under sustained inference. PSU sizing matters.

The right way to think about this rig class: it's the first tier where Inkling stops being an experiment and starts being usable, even if it isn't fast. If you want interactive speeds, plan on 2–4 flagship GPUs or a Mac Studio Ultra 192 GB. That is the practical floor for interactive local Inkling in mid-2026.

What to try first

• If you want to see Inkling in action without spending money, most third-party AI-search interfaces (Perplexity, You.com, several others) have added Inkling as a selectable model within days of release. Ask it a hard multi-step reasoning question that trips a lesser model and compare the trace.
• If you have a small cluster of consumer flagship GPUs (2× RTX 5090, 4× RTX 4090, or better), pull the fp8 checkpoint from Hugging Face and try llama.cpp's expert-offload branch. Community reports suggest 6–12 tok/s is achievable with careful tuning.
• If you have a datacenter-class node, follow Moonshot's reference vLLM configuration — the released deployment recipe is straightforward and produces the expected 40–70 tok/s throughput.

Whatever route you take, keep an eye on the community fine-tuning work that's already underway. Inkling's base model is strong; a Reddit-community-flavored instruct-tune, a coding-specialization tune, or a small-context latency-optimized tune could each dramatically change the calculus of what "self-hostable" means over the coming months.

Citations and sources

• Artificial Analysis — Inkling model page
• The Decoder — coverage of the Inkling release
• TechPowerUp — RTX 3060 spec reference
• llama.cpp on GitHub — MoE expert-offload work

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.