Qwen3.6-27B at 72 Tok/s on RTX 3090: What's Real

Search results and forum threads have been circulating a specific claim: Qwen3.6-27B running through vLLM on a single RTX 3090, natively on Windows, at 72 tokens per second — no WSL, no Docker. It's an appealing number: fast enough for real-time chat, on a 24GB card, without touching a Linux subsystem. It's also a number that couldn't be traced to a primary source, and parts of the setup it implies — a portable one-click native-Windows vLLM installer — don't match what the vLLM project currently documents. This piece separates what's confirmed from what's circulating, and lays out the setup path that's actually supported today.

The short answer

A widely shared claim puts Qwen3.6-27B on an RTX 3090 at 72 tokens per second through vLLM, running natively on Windows. No vLLM release note, GitHub discussion, or benchmark writeup we could locate attributes that figure to a specific build, quantization level, or context length. Treat it as an unverified community number, not a confirmed benchmark.

What is confirmed: the vLLM project documents Linux as its primary supported platform, directing Windows users toward WSL2 or Docker. First-party native Windows support isn't part of vLLM's official install matrix as of this writing. That doesn't mean nothing runs on bare Windows — PyTorch's CUDA build works natively, and unofficial forks of vLLM-adjacent tooling exist — but "native Windows vLLM" is currently a self-maintained path, not a supported one.

If the goal is simply getting Qwen3.6-27B running on a 24GB RTX 3090 as fast as possible, WSL2 matches vLLM's actual support surface today and avoids depending on unofficial patches of unknown provenance.

What vLLM on Windows actually supports right now

Source: vLLM installation documentation.

Given that table, any "portable launcher" claiming to get vLLM running natively on Windows should be treated the way you'd treat any unofficial installer requesting GPU and filesystem access: verify the source before running it. Prefer scripts published under the official vllm-project GitHub org or linked directly from vLLM's own docs over unaffiliated third-party executables.

Qwen3.6-27B on a 24GB card: what's realistic

Qwen's models are published on Hugging Face under the Qwen org. A 27B-parameter dense model at full precision (BF16/FP16) needs roughly 2 bytes per parameter for weights alone — around 54GB before the KV cache, well past a single RTX 3090's 24GB of GDDR6X memory. Fitting a 27B model on one RTX 3090 means quantization isn't optional:

These are standard quantization-math estimates, not a SpecPicks-run benchmark — actual footprint varies by quantization method, group size, and requested context length. The practical implication: a 72 tok/s figure, if real, almost certainly describes a heavily quantized build at a modest context length rather than full-precision inference. Throughput at that setting varies by prompt length, batch size, and how much of the 24GB is left for the KV cache after weights load — "varies by workload" is the honest answer where a specific number isn't independently sourced.

For a lower-VRAM comparison point on a still-current card, SpecPicks' guide to running Qwen 3.6 27B on a 12GB RTX 3060 walks through the same quantization tradeoffs at half the memory budget, and the Ollama RTX 3060 tok/s roundup covers how model size scales against a 12GB card more broadly.

Setting up vLLM on Windows: the supported path

1. Install WSL2. Follow Microsoft's official WSL install docs — wsl --install sets up an Ubuntu environment on a modern Windows 11 build.
2. Install NVIDIA's WSL-compatible driver on the Windows host (not inside WSL) — a single driver exposes the GPU to the WSL2 environment. Get it from NVIDIA's CUDA downloads page.
3. Inside WSL2, install vLLM per the official docs — a Python virtual environment plus pip install vllm covers most current GPUs, RTX 3090 included.
4. Pull Qwen3.6-27B weights from Hugging Face in a quantized format (GPTQ, AWQ, or GGUF, depending on which quant path vLLM supports) rather than the full-precision release.
5. Launch with a constrained --max-model-len to leave VRAM headroom for the KV cache — the single most common fix for "CUDA out of memory" errors on a 24GB card.

For a runtime that installs natively on Windows without WSL, llama.cpp on a 12GB RTX 3060 with llama-bench and Ollama's model-size guidance for 12GB cards both cover GGUF-based setups — a meaningfully different tradeoff (broader native-Windows support, typically lower peak throughput than a Linux-native vLLM build) worth weighing against the WSL2 route above.

Troubleshooting

"CUDA out of memory" on load or first request. Almost always means the requested context length (--max-model-len) plus KV cache overhead exceeds what's left after the quantized weights load. Lower --max-model-len, drop to a smaller quantization (Q4 instead of Q5/Q6), or close other GPU-resident applications competing for the same 24GB.

Driver mismatches inside WSL2. The GPU driver lives on the Windows host, not inside the WSL2 distro — installing an NVIDIA driver inside WSL2 is unnecessary and can conflict with the host driver. Update via Windows, not apt inside the Linux environment.

Confirming the GPU is actually visible. Run nvidia-smi inside WSL2 before troubleshooting anything vLLM-specific — if it doesn't report the RTX 3090, the problem is the WSL2/driver bridge, not vLLM or the model.

RTX 3090 vs. AMD for local LLM inference

vLLM's CUDA-first design gives an RTX 3090 broader, more mature backend support than AMD equivalents like the RX 6800 XT, where ROCm compatibility for vLLM is narrower and less consistently maintained. For AMD-specific throughput and accuracy comparisons against Qwen3.6-27B, SpecPicks' Qwen3.6-27B vs. Coder-Next benchmarks on AMD rigs is a more direct reference point than extrapolating from Nvidia-side vLLM numbers.

For a second Nvidia data point on the same 24GB card, see Qwen 3.6 27B throughput with Luce DFlash on a single RTX 3090, which covers a different runtime/config than the vLLM path described here. For a broader look at where local inference tops out on smaller hardware, can a Raspberry Pi 4 8GB run a local LLM in 2026? is a useful low-end contrast, and Intel IPEX-LLM + Ollama on Arc vs. the RTX 3060 rounds out the cross-vendor picture.

Should you bother with a "portable" install?

The appeal of a USB-portable, no-install launcher is real — it avoids touching system Python, drivers, or registry entries. But portability and native-Windows vLLM support are two separate problems, and only one of them is currently solved by the official project. A more reliable version of "portable" on Windows today is exporting a configured WSL2 distro as a single .tar file and importing it on another machine, which keeps you on vLLM's supported Linux-based path rather than an unverified native Windows build. If you do encounter a third-party "portable vLLM for Windows" installer, verify it against the publisher's published checksums and source repository before running it — the same due diligence you'd apply to any executable requesting GPU and filesystem access.

Related reading

• llama.cpp on a 12GB RTX 3060: llama-bench setup and measured tok/s
• Run Qwen 3.6 27B locally: VRAM, quant & tok/s on a 12GB RTX 3060
• Ollama on the RTX 3060 12GB: model sizes and tok/s for 2026
• Qwen3.6-27B vs. Coder-Next: local code accuracy and VRAM on AMD rigs
• Qwen 3.6 27B at 2x tok/s: Luce DFlash on one RTX 3090
• Can a Raspberry Pi 4 8GB run a local LLM in 2026?
• Intel IPEX-LLM + Ollama on Arc: setup, tok/s, and the RTX 3060 reality

Frequently asked questions

Does vLLM officially support native Windows, without WSL or Docker? No. vLLM's documented install matrix centers on Linux, with WSL2 or Docker as the supported paths for Windows users. Anything marketed as "native Windows vLLM" today runs on unofficial patches or forks, not the mainline project's supported configuration.

Is the "72 tok/s" figure for Qwen3.6-27B on an RTX 3090 accurate? It couldn't be traced to a primary, citable source. It's plausible as a quantized-model figure at a specific context length and batch size, but without a documented build and test configuration behind it, treat it as an unverified community claim rather than a confirmed benchmark.

How much VRAM does Qwen3.6-27B need on a 24GB card? Full-precision (FP16/BF16) weights alone would need roughly 54GB, well beyond a single RTX 3090. A 4-bit quantized build (GPTQ, AWQ, or GGUF) brings the weight footprint down to roughly 14–16GB, leaving headroom for the KV cache — the practical path for fitting a 27B model on 24GB.

What's the safest way to get vLLM running on a Windows gaming PC today? Install WSL2, install NVIDIA's WSL-compatible driver on the Windows host, and run vLLM inside the WSL2 Linux environment per the official docs. That's the path vLLM actually documents and supports, versus relying on an unverified native-Windows installer.

Can I run Qwen3.6-27B from a USB drive or portable install? There's no confirmed, official portable native-Windows vLLM build. A more reliable version of "portable" is exporting a configured WSL2 distro as a single file and importing it on another machine, which keeps you on vLLM's supported Linux-based path.

How does an RTX 3090 compare to an AMD RX 6800 XT for this workload? vLLM's CUDA-first architecture gives Nvidia cards broader, more consistently maintained backend support; AMD's ROCm path for vLLM is narrower. See SpecPicks' AMD-focused Qwen3.6-27B comparison for workload-specific numbers rather than extrapolating across architectures.

Citations and sources

• vLLM project on GitHub
• vLLM installation documentation
• Qwen models on Hugging Face
• NVIDIA GeForce RTX 3090 specifications
• Microsoft WSL install documentation
• NVIDIA CUDA downloads

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.