Frontier Models Cheat on Security Tests: Why You Should Eval Locally

Yes — recent evaluation work has surfaced that frontier LLMs sometimes "cheat" on published cybersecurity benchmarks by having seen the answers during training or by pattern-matching test artifacts, which means public leaderboard scores overstate how well those models actually detect vulnerabilities in your codebase. The fix is running your own held-out evaluations on models you control, and a 12GB local rig built around an NVIDIA RTX 3060 12GB is enough hardware to do it.

Why "cheating" on security benchmarks is a real, boring problem

Benchmark contamination in language models is old news, but the security-evaluation community got a particularly ugly reminder recently: several widely-cited cybersecurity benchmarks appear to have been at least partially memorized by frontier models during pretraining, which inflates their reported scores by amounts that can flip a ranking. "Cheating" is uncharitable — no one at a frontier lab is deliberately tuning a model to game a public CTF — but the practical effect is the same: the benchmark number does not tell you how the model will do on a novel vulnerability in code it has never seen.

For a security team, this is not an academic concern. If you were going to bet CI budget on a specific model's vulnerability-detection performance, you now have to assume the public score is an upper bound, not a fair estimate. The only way to get a real number is to evaluate the model on a held-out set from your own repositories, using tasks the model has demonstrably never seen. That is straightforward to do — but it works better when the model runs locally, because then you know exactly what happened during evaluation and no one else sees your held-out set.

This piece walks through why local evaluation matters, how to set up a modest eval harness, and what hardware you need to run it. If you own the model weights and a card with 12 GB of VRAM, the whole thing fits inside a normal engineering workflow.

Key Takeaways

• Public cybersecurity benchmarks are known to leak into pretraining data, which inflates frontier-model scores.
• You cannot trust a leaderboard number as an estimate of performance on your codebase — run your own eval on held-out data.
• Local eval on an RTX 3060 12GB hosting a 7-13B model is enough for meaningful measurement.
• Pair the GPU with a Ryzen 7 5800X host and a fast NVMe like the WD_BLACK SN770.
• The value of local eval is uninflated numbers — you learn what the model actually does on your problem.

What "cheating" looks like in practice

The failure mode most often reported in 2026 security-eval writeups is that frontier models score suspiciously well on flagship benchmarks like known-CVE test suites, but their advantage evaporates when the same class of task is presented with fresh, unseen examples. The reason is exposure: pretraining corpora scraped from the open web include huge amounts of security tutorial content, CTF walkthroughs, CVE descriptions with worked examples, and even the full text of the benchmarks themselves. A model that has seen "look at this snippet, identify the SQL injection, the answer is on line 4" during pretraining will, unsurprisingly, do very well when asked that at eval time.

Small task-specific models trained on curated security datasets — the kind Cisco published recently — do not necessarily "cheat" less; they may cheat differently. The difference is that when you run the eval yourself against a private held-out set, you can compare apples to apples: the same set of files, the same prompt structure, the same scoring rubric. That is the only way to get a defensible number.

Why local eval fixes this

Three properties of local evaluation matter:

Isolation. The held-out set never leaves your infrastructure, so you can be certain it is not accidentally added to a future training corpus by anyone. On a cloud API, you can add "please do not train on this" to your account settings, but you cannot audit whether that flag was honored.

Repeatability. With local weights, the same input produces the same output tomorrow, next week, and next year. Frontier APIs update behind the scenes; a benchmark run today is not necessarily comparable to a run in six weeks. When you are trying to measure a real effect, that noise floor matters.

Cost per experiment. Local eval on a card you own has zero marginal cost per inference. If you want to try seven different prompt formats, five different quantizations, and three different scoring rubrics, the cost is your time, not your budget. This is the single largest quality-of-life improvement for research-shaped work.

What hardware you need to eval a small security model

An RTX 3060 12GB is enough. Per TechPowerUp's specifications, the card has 3,584 CUDA cores and 12 GB of GDDR6 at 360 GB/s over a 192-bit bus. For a 7-13B quantized model at q4 or q5, that is comfortable throughput — 30-70 tokens per second in the decode phase, depending on model size and runtime — and enough VRAM for a 4-8k context window.

Pair it with a modern host: an AMD Ryzen 7 5800X provides eight cores of prefill support and enough single-thread performance to keep the GPU fed. Storage matters more than people expect for eval workloads — you will be loading and unloading many model checkpoints as you try different quantizations, so a fast NVMe like the WD_BLACK SN770 turns model-swap latency from painful minutes to painless seconds. 32 GB of DDR4 is comfortable; 16 GB is the floor if you never plan to offload.

A minimum viable eval harness

The shape of a useful security eval is:

1. Held-out set. 100-500 tagged snippets from your own code, chosen so at least half contain a known vulnerability of a specific class (SQL injection, path traversal, missing auth check, etc.). Human-labeled ground truth is the entire point.
2. Fixed prompt template. The same wording, in the same order, for every model you evaluate. If you change the prompt between models, you are measuring prompt sensitivity, not model quality.
3. Structured output. Ask the model to return JSON with a verdict and an explanation. Grading a free-form response is a research project of its own; grading a structured verdict is a one-liner.
4. Metric. Precision at high recall is usually what matters for security triage: you want a model that flags almost all real vulnerabilities without drowning the reviewer in false positives.
5. Baseline. Include a random-verdict baseline and a "flag everything" baseline. If the model's precision at high recall does not beat those, the model is not doing anything useful.

Frameworks like llama.cpp make the mechanical part trivial: load a GGUF model, iterate over prompts, capture responses, compute the metric. The intellectual work is designing a held-out set that actually reflects your bug distribution.

What to expect from a 7-13B security model on your held-out set

Rough expectations, absent a specific model in hand: a well-tuned small security model that scores well on public benchmarks typically retains most of its performance on a novel held-out set of code that resembles its training distribution — think common web-app languages and vulnerability classes. It loses more performance on unusual stacks (rare languages, embedded firmware, novel weakness classes) and on prompts that require multi-file reasoning.

If your codebase is a mid-sized Python or JavaScript service with a normal distribution of bug classes, expect a 7-13B security model at q5 to give you real signal: precision in the 60-80% range at recall levels around 70-80%, depending on the difficulty of your test set. Those are numbers that let you build a triage-assist pipeline. They are not numbers that let you replace human review.

Spec + benchmark table: single-node local eval throughput

Times assume single-request eval with a 200-token output budget per snippet. Batching multiple prompts per forward pass can substantially improve throughput on this card if your runtime supports it.

Cost math: local eval vs API eval

A 500-snippet eval with a 200-token output budget is 100,000 tokens of output plus roughly 500,000 tokens of input (assuming 1,000-token snippets). At a $3/$15 per million tokens frontier API pricing shape, that is roughly $3 per eval run. That sounds cheap, and for a one-shot experiment it is. It is not cheap for research-mode iteration where you may run 50-100 eval runs while you tune the harness, and it certainly is not cheap if you plan to continuously re-eval every few weeks against updates.

The local hardware amortization for the RTX 3060 + Ryzen 7 build is roughly $750 up front. At $3 per eval run, that is 250 runs before the local box is cheaper, ignoring electricity. Most teams doing serious eval work cross that threshold in a quarter.

Common pitfalls

• Not shuffling your held-out set between eval runs; you end up chasing noise that is really an order effect.
• Grading free-form answers by hand; use structured JSON output and grade programmatically.
• Comparing precision without controlling recall; a model that says "no vulnerability" every time will have 100% precision on the negatives.
• Trusting a single eval run; run 3-5 with different random seeds if the model supports temperature > 0, average the results.

Bottom line: run your own evals

Public benchmark scores are not a lie, but they are a ceiling, and for cybersecurity workloads specifically that ceiling is above what you will see on your own code. Running local evaluation on a 12GB rig gets you defensible numbers, complete isolation of your test data, and the freedom to iterate cheaply. If you were going to make a purchase decision on the strength of a leaderboard, the RTX 3060 + Ryzen 7 build pays for itself in avoided bad decisions.

Related guides

• Cisco's Open Security LLMs: Local Vulnerability Scanning on an RTX 3060 — the model you probably want to start with.
• Best Budget Local-LLM Box: RTX 3060 12GB + Ryzen 5 5600G — a cheaper host if you can drop cores.
• Llama.cpp vs Ollama on an RTX 3060 for Single-User Chat — pick a runtime for eval.

Citations and sources

• TechPowerUp — GeForce RTX 3060 specifications
• Cisco Blogs (security channel)
• ggml-org/llama.cpp on GitHub

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.