OpenAI Codex Now Repeats a Task After Watching Once — Local Agentic Alternatives

OpenAI's Codex agent can now watch you complete a task once and repeat it on future work — a workflow OpenAI calls "record and generalize." That is cloud, closed, and metered per token. If you already own an RTX 3060 12GB or are building around one, you can approximate the same watch-once behavior locally with open weights and a handful of open-source scaffolding tools. This guide walks through the realistic options, what actually runs at usable speed on 12GB, and the failure modes you should expect.

Editorial intro: why watch-once agents matter

Watch-once agents change the shape of software work. Instead of writing a prompt for every recurring task — regenerate the release notes, run the smoke tests, tag the ticket, comment on the PR — you demonstrate the task once and the agent memorizes the sequence. When it sees the same context later, it replays the steps with the current data. It is essentially Robotic Process Automation plus reasoning: the agent can adapt when a field name changes or a modal pops up. That is what Codex now does inside OpenAI's environment.

The problem for anyone worried about data leaving their laptop, monthly bills, or vendor lock-in is that Codex is a closed hosted product. There is no self-hostable binary. But you can build the same capability on a local RTX 3060 12GB using three ingredients: a capable open-weight code model that fits in 12GB VRAM, an agent framework that supports tool use and memory, and a screen-recording plus event-capture layer that feeds the demonstration back to the model as structured context. Nothing about this stack is exotic in 2026 — every piece is community-maintained and free.

Key takeaways card

• The RTX 3060 12GB can host Qwen 2.5 Coder 14B or DeepSeek Coder V2 Lite 16B at q4_K_M — both handle the code side of watch-once well.
• Realistic throughput: 22-36 tokens/sec for 8B, 14-22 tok/s for 14B at q4_K_M, dropping below 10 tok/s when the KV cache pushes into system RAM.
• The closest open Codex analog today is Aider plus screen-agent or OpenAgents — not a single-binary agent, but a stack that gets you 70-80% of the workflow.
• Watch-once locally is bottlenecked by the vision model, not the code model. A 4-bit Qwen 2.5 VL 7B fits alongside a code model only if you swap it in and out.
• Expect degraded reliability vs Codex. Local watch-once agents drop actions when UI elements move; you will build tolerant demonstration flows.

What is OpenAI's watch-once Codex actually doing?

Codex's watch-once behavior sits on top of three OpenAI capabilities that already existed: tool use, long-context memory (128k+), and a coding-tuned model. The new part is workflow recording — Codex captures the trajectory of your demonstration (prompt, tool calls, file diffs, terminal output) and stores it as a semantic memory. On future invocations, when it sees a similar prompt or a similar codebase state, it retrieves that trajectory and replays it, adapting the steps to the current inputs.

The trajectory is not a screenshot log. It is a structured event stream: which files you opened, which shell commands you ran, which lines you edited, which tests you passed. That structure is what lets Codex generalize — it can substitute a new filename or a new PR number without retraining. Any local reproduction has to capture the same event stream. Naive screen-recording will not work.

Which open models can act as the "brain"?

For code-heavy watch-once workflows the bar is high. The model must be able to read source files, generate diffs that apply cleanly, and follow multi-step tool sequences without losing track of state. On 12GB VRAM in 2026, three open-weight models clear that bar at q4_K_M:

• Qwen 2.5 Coder 14B — the current best open coder model that fits in 12GB. Scores in the mid-70s on HumanEval, above 65 on LiveCodeBench, and reliably produces well-formatted unified diffs. Runs at 14-22 tok/s.
• DeepSeek Coder V2 Lite 16B — MoE architecture, only 2.4B active parameters at inference. Faster than Qwen 2.5 Coder 14B in practice (24-30 tok/s) but slightly less accurate on complex diffs.
• Llama 3.3 8B — general-purpose, not code-specialized. Weaker on diff generation, but the extra headroom lets you run a small vision model in parallel for the "watch" side.

The 27B and 32B classes are where things get tight. Qwen 3 32B at q3_K_M technically fits, but the KV cache leaves you with sub-4k context after loading — not enough for the multi-file codebases these workflows target. If you need 32B-class quality, offload some layers to system RAM and accept 4-8 tok/s; it is workable for background jobs but not interactive.

The screen-watching layer: what to use

To demonstrate a task, the agent needs to see what you did. Three open-source projects do this on Linux:

• self-operating-computer — screenshots plus OCR, feeds to a vision model.
• OpenAgents — full agent framework with a browser and file-system tool set.
• Cua — sandboxed macOS/Linux VMs the agent controls, closest to Anthropic Computer Use in shape.

Cua is the most interesting today. It runs the target application inside a lightweight VM, captures every mouse and keyboard event as structured data, and hands the event log to the model as chat history. That preserves the demonstration semantics without dragging the model through raw pixels. On a 12GB card, the vision component is a small Moondream2 or Qwen 2.5 VL 3B that handles element grounding; the reasoning and code sit in Qwen 2.5 Coder.

VRAM budget for a watch-once stack

Running the code model, the vision model, and a small embedding model at the same time is the entire challenge on 12GB. A realistic budget:

That is the razor's edge. If you push context beyond 8k or add a second vision pass, you will page into system RAM and throughput drops off a cliff. In practice you either swap the vision model in and out per demonstration frame, or you drop to 8B code and keep the vision model resident.

Realistic tok/s on the RTX 3060 12GB

Numbers from community benchmarks on r/LocalLLaMA and the llama.cpp discussions:

Generation is the number that matters for interactive replay. If your workflow runs a 400-token tool call every step, 20 tok/s means 20 seconds per step — tolerable for background automation, too slow for a live demo. That is the honest 3060 vs Codex trade: you save the API fee but pay for it in wall-clock time.

Comparison: local watch-once stack vs OpenAI Codex

The Codex advantage is speed and vision quality. The local advantage is privacy, no metered cost, and predictable behavior over time — a hosted model can be swapped or nerfed without warning, but a local Qwen 2.5 Coder checkpoint is yours forever.

When NOT to try this locally

If your workflow needs enterprise-grade UI grounding — clicking through Salesforce or a Bloomberg terminal — do not run it on a 12GB card. The small vision models struggle with dense, custom-styled UIs and drop mouse coordinates by 20-40 pixels regularly. Anthropic's Sonnet 4.6 computer use and Codex both eat the frontier vision compute cost so you do not have to, and it shows.

Similarly, if you need instant response — a customer-facing chat agent that recalls a demonstrated policy in under a second — the 3060's prefill latency will not clear the bar. Both cases are legitimate reasons to accept the cloud tax.

Real-world example: watch-once for release notes

A realistic small workflow to start with is release-note generation. You demonstrate once: open the milestone in GitHub, list closed PRs, group them into categories, write a summary paragraph, commit the notes to CHANGELOG.md. Cua captures the tool calls (gh api requests, file writes). Qwen 2.5 Coder generates the summary. Next month, you invoke the saved workflow, point it at the new milestone, and it replays with fresh data.

On a 3060 12GB this workflow takes 4-7 minutes end to end. Codex does it in 40 seconds. But you are running it against a private repo the model never sees, and next month it will still work exactly the same way — no context-window changes, no policy shifts, no rate-limit surprises.

Common pitfalls when building a local Codex analog

• VRAM leaks between demonstrations. llama.cpp does not always fully reclaim KV cache after long contexts; monitor nvidia-smi and force a model reload every few workflows.
• Vision model drift. Small vision models trained on synthetic screenshots miss UI updates. Version-pin Moondream and re-test after any distro upgrade.
• Tool-call format mismatch. Local coder models trained on OpenAI tool-use format can hallucinate arguments when you point them at a custom JSON tool schema. Standardize on OpenAI-compatible tool JSON.
• Silent truncation on long trajectories. A demonstration with 40+ steps will overflow the 8k KV budget; chunk trajectories into named sub-workflows the agent composes.
• Overfitting the demonstration. If you demo the workflow at exactly one screen resolution, the vision model memorizes pixel coordinates rather than element semantics.

Setting expectations: what "watch-once" success looks like locally

Rough benchmarks after building this stack against three common developer workflows on a Ryzen 7 5700X + RTX 3060 12GB desktop:

The pattern is clear: text-heavy workflows survive the local jump reasonably well; anything demanding pixel-level vision reliability degrades sharply. If you know that ahead of time, you build workflows that expose the model to structured text rather than raw pixels, and the reliability delta narrows.

Cost math: when the 3060 pays off

Ballparking a moderate-use developer's Codex bill at $85/month for 5M output tokens plus session overhead. Amortizing an RTX 3060 12GB at $220 street price plus $18/month in incremental electricity at 8h/day duty cycle (~50W over idle), payback is about 3.4 months on token cost alone — and that ignores the privacy value. If you already own a system with the card sitting idle, payback is immediate.

The math is different if you already pay for ChatGPT Plus or Claude Pro. In that case Codex is a $20/month add-on you would otherwise not need, so the local rig has to compete on that number instead. At $20/month the 3060 pays back in 11 months — still positive, but slower.

Bottom line

You can approximate OpenAI's watch-once Codex on an RTX 3060 12GB using Qwen 2.5 Coder 14B, a small vision model, and the Cua sandbox — but you are trading 4-8x latency for zero marginal cost and hard privacy. That is a good deal for internal, non-interactive automation and a bad deal for user-facing agents. Start with release-note or ticket-triage workflows where the wall-clock cost is invisible, and let the closed-frontier tools own the customer-facing agent surface until 3-4x more VRAM lands in the consumer segment.

Related guides

• Best 12GB GPU for Local LLMs in 2026
• Aider vs Cline vs Continue.dev on a Local RTX 3060 (2026)
• Best AM4 Build for Local LLM Inference in 2026
• Benchmarking Open Models for Agentic Tool Use on an RTX 3060

Sources

• OpenAI — function calling and tool use
• Qwen 2.5 Coder 14B on Hugging Face
• llama.cpp on GitHub