After the Mythos Cyber-Ops Report, Why Run AI on an Air-Gapped Local Box

An air-gapped local LLM rig answers the question "where does my prompt actually go?" with a simple, auditable boundary: nowhere. The Mythos Cyber-Ops incident this week is a reminder that any model behind an API is also behind an attacker. For users handling internal documents, client data or anything regulated, a local box on isolated power and isolated network beats every promise of "we don't train on your data."

Why this matters right now

The Mythos Cyber-Ops disclosure — and the broader pattern it sits inside — has pushed a lot of mid-sized firms to re-read their AI vendor contracts. The pattern is familiar: prompts and uploaded files routed through inference providers, retained for some span, occasionally used for evaluation. Almost every commercial term of service permits this in some form. For most users it is fine. For the subset of users whose data is genuinely sensitive — legal discovery, security incident response, M&A drafts, medical records, code with embedded credentials — the right answer is to never let those tokens leave the building.

That subset is not a niche. It is a growing share of practical AI use, and the hardware to support it is now cheap. A ZOTAC Gaming GeForce RTX 3060 Twin Edge 12GB or MSI GeForce RTX 3060 Ventus 2X 12GB plus a Ryzen 7 5800X and a quiet Crucial BX500 1TB for the model library is enough to run useful Qwen-3 14B, Llama-3 8B and DeepSeek-R1 distill workloads at sensible throughput for one user. The rest of this synthesis is how you wire that into something defensible.

Key takeaways

• "Air-gapped" means physically isolated network and storage, not just an off switch on Wi-Fi.
• 12 GB of VRAM runs a useful 7B–14B model lineup at 4-bit quantization with comfortable headroom.
• The disk you copy models from matters as much as the GPU — provenance is part of the threat model.
• Boot media should be read-only or known-good; model weights should be hashed against an offline manifest.
• A tunneled "isolated except when I update it" rig is not air-gapped; treat it as connected.
• Don't conflate local with anonymous — log retention and physical access still apply.

What does "air-gapped" actually mean in 2026?

An air-gapped system is one with no concurrent network path to any system outside a defined trust boundary. The 1990s definition required physical separation of cabling and switches. The modern definition has to deal with Bluetooth, Wi-Fi 7, cellular modems baked into motherboards, and out-of-band management interfaces like Intel ME and AMD PSP.

The NIST SP 800-53 control family treats air gaps as a configuration plus operational discipline, not a single hardware property. In practice this means: the box has no enabled network interfaces during operation; updates arrive via a one-way transfer process (sneakernet of a verified medium, typically a write-once optical disc or a checksum-verified USB flash device that lives in a faraday bag between uses); the BIOS has integrated network controllers disabled at the firmware level; the OS is hardened to refuse driver loads for new network devices.

A laptop with the Wi-Fi switch flicked off is not air-gapped. It is convenient.

Can you actually run useful AI on an air-gapped local box?

Yes, with caveats. The model family that fits cleanly on a 12 GB card at Q4_K_M GGUF quantization, per the llama.cpp project's discussions, is:

• Qwen3-14B-Instruct (≈9 GB in Q4_K_M, ~22 tok/s on a 3060 12 GB)
• Llama-3.1-8B-Instruct (≈5 GB in Q4_K_M, ~38 tok/s)
• DeepSeek-R1-Distill-Qwen-14B (≈9 GB in Q4_K_M, ~18 tok/s)
• Phi-4-14B (≈9 GB in Q4_K_M, ~20 tok/s)
• Gemma-2-9B-it (≈6 GB in Q4_K_M, ~30 tok/s)

For most knowledge-worker tasks — drafting, summarization, code review, structured extraction — Qwen3-14B and Llama-3.1-8B are the daily drivers. Reasoning is the gap. R1-Distill closes much of it but with worse hallucination rates than the frontier hosted models. Per the Hugging Face Open LLM Leaderboard archive, the 14B-Q4 tier sits roughly where the GPT-3.5-class hosted models sat in 2023 — useful, not state of the art.

Spec table: minimum vs comfortable air-gapped LLM build (2026)

The "comfortable" build still lands under $1,200 used. The "minimum" build comes in near $850 used and runs a daily workflow without complaint.

Benchmark table: 4-bit local LLM throughput on the 3060 12 GB

Per community measurements collected in the llama.cpp repo and reproducible with llama-bench, single-user throughput on an RTX 3060 12 GB at Q4_K_M GGUF lands in this range:

A 4080 16 GB roughly doubles those numbers; an MI300X (datacenter-only) runs another 3–4× faster but is not the right tool for a one-user air-gapped box.

Provenance: how do you trust the weights you copied in?

This is where most "local LLM" articles handwave and most "air-gapped LLM" deployments fail in practice. The model is just a file. If you copied it from a network you do not control, you copied an artifact of unknown provenance. The verification chain that holds up under audit looks like:

1. Hash the upstream artifact on a connected build host the moment you download. The llama.cpp converter scripts emit deterministic outputs; published GGUFs from reputable mirrors carry SHA-256 sums.
2. Sign the manifest (file path, size, sha256) with an offline signing key.
3. Transfer via write-once media — DVD-R or a fresh USB flashed via dd from a known-good source — and burn the manifest onto the same medium.
4. On the air-gapped box, verify before mounting. The model is not loaded until the manifest checks out.

This is annoying. It is also the only chain a security auditor will accept for "we run inference on regulated material." A model swapped for a poisoned look-alike will pass every functional test until it doesn't.

OS, drivers, BIOS: what stays off

A clean air-gapped Debian 12 build for the 5800X + 3060 stack looks like:

• BIOS: AMD PSP enabled (you can't disable it cleanly on AM4), wake-on-LAN off, network controllers disabled, USB boot allowed only from one known port.
• Kernel: stock Debian, no out-of-tree drivers except the official NVIDIA proprietary driver matching your CUDA toolkit.
• No wpa_supplicant, no NetworkManager, no bluez. systemctl mask them.
• No remote management agents. No telemetry from Ollama; verify with tcpdump once on a connected mirror, then deploy.
• dnsmasq not installed; outbound DNS to nowhere is part of the gap.
• Logging stays local and on a separate partition that you can rotate to verified media.

The reusable AI tooling like Ollama and llama.cpp can run perfectly air-gapped — they don't phone home in normal use, but you must confirm that on a connected dry-run before deployment.

Common pitfalls

• Updates that punch the gap. "We'll just plug it in once a week to apt-get." That is a connected machine that pretends to be air-gapped. Either commit to the sneakernet update process or stop calling it air-gapped.
• USB the size of a small datacenter. Treat every USB device as a write-once artifact. The Stuxnet model still applies; a re-used thumb drive is a vector.
• Forgetting Bluetooth. Many AM4 motherboards ship with no Bluetooth, but if you used a board with onboard WiFi/BT, disable both at the firmware level.
• Mixing in user devices. A keyboard or webcam shared with a connected workstation is a side channel. Dedicate peripherals.
• Cloud-trained LoRAs. A LoRA you fine-tuned on Hugging Face inference and downloaded back to the box is a network artifact. Hash, manifest, ingest like any other weight.
• Logging telemetry on by default. Some inference servers expose Prometheus metrics on 0.0.0.0:9090 by default. Bind to localhost or disable.

Three worked examples — what the rig actually does day-to-day

Outside legal counsel handling discovery. The intake is hundreds of PDFs covering a single matter. The rig runs Qwen3-14B-Instruct with a llama.cpp server and a thin local UI. The lawyer drops PDFs into a watched folder; a script runs OCR, chunks the text, and feeds it to the model with structured-extraction prompts ("party names, dates, relevant clauses"). The output goes to a local SQLite database that never leaves the box. No prompt or document ever touches a cloud provider. Throughput on the 3060 12 GB is roughly 800 pages per hour of useful extraction — slow versus hosted, fast versus billable hours.

Security incident response. During a live incident the team needs to summarize a stream of log excerpts, identify likely IOCs, and draft customer comms. A connected box would mean exfiltrating live attacker traces to a third party. The air-gapped rig runs Llama-3.1-8B for fast summarization and DeepSeek-R1-Distill-14B for slower reasoning over short windows. The team's runbook is "paste the redacted excerpt into the local UI." Throughput is plenty for one or two analysts; the value is the boundary, not the speed.

Internal codebase code review. Engineering wants AI-assisted review of patches touching auth and crypto code, but the policy forbids sending those patches to any external provider. The rig runs Qwen3-14B as a git pre-push hook host — the diff is piped into a structured-review prompt, the model returns findings, the engineer reviews and pushes. Latency is 4–12 seconds per patch on the 3060, well below frustration threshold.

In all three cases, the rig replaces neither the team nor the hosted models — it covers the slice of work where data sensitivity is the gating constraint.

When NOT to build this

• Your data is not actually sensitive. Most code, marketing copy and personal notes don't justify the operational burden.
• You need the frontier. A 14B Q4 local model is not Claude or GPT — if you need cutting-edge reasoning, you have to pay for the meter and accept the data flow.
• You don't have the operational discipline. An air-gapped box you forget to update is a security artifact pretending to be a productivity tool.

For everyone else, the 12 GB RTX 3060 + Ryzen 7 5800X + Crucial BX500 1TB build is a defensible, quiet, capable rig. It will not impress a benchmark thread on Reddit. It will pass an auditor's questions about where the data went.

Bottom line: should you build an air-gapped LLM rig in 2026?

Build it if you handle regulated data, you have the operational chops to keep the gap intact, and you want a defensible, auditable boundary for sensitive inference. Skip it if you need frontier reasoning, or your data sensitivity doesn't justify the discipline. The hardware floor is the ZOTAC RTX 3060 12 GB and MSI RTX 3060 Ventus 2X 12 GB, paired with the Ryzen 7 5800X and a Crucial BX500 1TB SATA SSD for the model library — under $1,000 used and entirely capable.

Related guides

• Local Image and Video Generation on a 12 GB RTX 3060 — same hardware, different workload.
• vLLM on an RTX 3060 12 GB: Is It Worth It for Single-User Chat?
• Ryzen 7 5800X vs 5700X for Gaming and Local AI

Citations and sources

• llama.cpp on GitHub
• NIST SP 800-53 control family
• KrebsOnSecurity — recent industry incident coverage

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.