Qwen on an RTX 3060: Why 50 Tokens per Second Is Not a Coding-Agent Verdict

A model in the 35B class, an RTX 3060 with 12GB of VRAM, 16GB of system RAM and approximately 50 tokens per second: that is the claim in the shared X post. It is interesting—but it leaves out the most useful follow-up. [1]

The original developer later reported that this distill solved 10 out of 20 tasks in his coding-agent test, with most failures involving prolonged reasoning or loops. Fast output and successful autonomous work were not the same result. [7]

I traced the repost to Doğukan (@DogukanUrker), inspected the linked video, checked the actual Empero model cards and file metadata, and read the developer’s later benchmark posts. I did not download the model weights, run this configuration or reproduce the speed or task scores. This is a source-and-evidence investigation, not my own RTX 3060 benchmark.

1. First correction: this is Empero’s community distill

The checkpoint is empero-ai/Qwen3.8-35B-A3B-Distill, not a checkpoint published under the official Qwen organization with that name. Empero describes it as distillation from Qwen3.8 teacher models into the Qwen3.6-35B-A3B architecture. That is the developer’s stated training provenance, not an independently audited training run. [3]

The distinction matters: a community distill can inherit an architecture without inheriting every capability or benchmark result of its teachers. “Qwen3.8” in this checkpoint’s name is not proof of official release status or parity with a larger Qwen3.8 model.

The reviewed sources are pinned here:

• Empero model card/config: bcc2dbe2f21b213625df2dc1a5a690212373af07.
• Empero GGUF repository: b1f9d1dcc3de8aa867669b0ab919384aeeb9b8d5.
• Official Qwen3.6 base-model card: 995ad96eacd98c81ed38be0c5b274b04031597b0.

The base is a sparse Mixture-of-Experts model: approximately 35B total parameters, around 3B activated per token. The model card describes a hybrid attention architecture rather than a conventional full-attention layer at every depth. [3][4][5]

Active parameters reduce per-token computation; they do not reduce the entire stored checkpoint to the size of a 3B model. You still have to place the weights and runtime state somewhere.

2. The actual timeline changes the story

The original speed/configuration post is dated September 16, 2026. It reports approximately 50 tokens/s decoding, 550 tokens/s prompt processing and a configured 262K context. Crucially, it says the coding-agent comparison and a speed-versus-context graph are next steps, not completed evidence in that post. [2]

On September 21, the developer published his 20-task Pi Agent coding result. He reported:

• Qwen3.8-27B at IQ2_XS: 16/20.
• Base Qwen3.6-35B-A3B: 16/20.
• Ornith-1.5-35B: 15/20.
• Empero Qwen3.8-35B-A3B distill: 10/20. [7]

For the distill, he attributed nine of its ten failures to runaway reasoning: six hit the output cap and three were stopped for looping. His conditional “10/11 when it started editing” observation should not replace the end-to-end 10/20 outcome. Runs that never reach the editing stage still count against an agent workflow. [7]

An October 27 follow-up reports a defect in a hidden benchmark test and says the run history was regraded. The author says only a Bonsai result changed. That correction matters, but it does not turn the earlier distill result into a newly reproduced independent score. [8]

The pinned BenchKit documentation corroborates the nature of the hidden-test defect: an earlier grading revision enforced an exact token layout that the underlying issue did not require. It does not independently establish every historical regraded score. [12]

I also inspected the complete BenchKit tree at 5945b05d21b8717fed3d872652a5b44d9efb2c1e. I did not find the historical Empero run export, per-task traces or a manifest tying the September test to exact model, serving and harness revisions in that snapshot. Its /results directory is ignored. This is an evidence boundary for the inspected tree, not proof that such records exist nowhere else. Current harness options cannot fill in the missing historical limits or loop-kill procedure. [13][14]

These are the developer’s measurements on a small custom suite. The models used different quantizations, and the setup is a practical hardware-budget comparison—not a controlled experiment isolating one architectural variable. The report also mentions continued training and a planned rerun: this is a historical checkpoint observation, not a demonstrated score for every later Empero release. Do not convert it into a universal ranking of all coding models. [7]

3. What the video actually demonstrates

The clip shows a chat prompt asking for a simple full-stack Flask app. The response generates code and instructions for a task manager, including a Flask backend, SQLite storage and browser-side operations.

At the end, the visible footer shows 49.99 t/s, with a tooltip labelled “Generation speed.” It also displays 2,334 tokens and 46s. I can verify that those values appear in the video; I cannot independently attest the server’s timing method or the exact hardware from the browser screen alone.

The clip does not show:

• An agent editing a real repository through tool calls.
• The generated application being launched and used.
• Tests being executed successfully.
• A filled 262K-token conversation or a context-length sweep.
• GPU/CPU memory traces, paging activity or repeated-run statistics.

The desktop/browser appearance does not establish where the inference server is hosted. The hardware and approximate 550 tokens/s prefill figure remain the original developer’s report, not values I independently measured. [2]

That is not an accusation that the clip is fabricated. It is the narrower conclusion the available evidence supports: a plausible, visibly fast code-generation demo is not itself a coding-agent evaluation.

4. The model is not sitting entirely inside 12GB VRAM

The GGUF repository lists the Q4_K_M file at approximately 21.713 decimal GB. The file API reports 21,713,462,944 bytes, about 20.22 GiB. That is already larger than the card’s stated 12GB VRAM, before working buffers and cache. [4][11]

The original command combines GPU offload with CPU-resident MoE weights:

llama-server \
  -m Qwen3.8-35B-A3B-Q4_K_M.gguf \
  -ngl 99 --n-cpu-moe 26 -c 262144 -fa on --jinja \
  -np 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --reasoning on

This is the developer’s reported configuration, preserved for inspection—not a command I ran or a guaranteed setup for every 3060 machine. [2]

In the llama.cpp option implementation I inspected, --n-cpu-moe 26 keeps the MoE weights of the first 26 layers on the CPU. It does not simply mean that 26 complete transformer layers, including every operation, move to the CPU. -ngl 99 requests GPU offload while the MoE override changes where those expert weights reside. [10]

Other relevant choices:

• Q4_K_M quantizes the weights; it does not eliminate their storage or bandwidth requirements.
• -fa on requests Flash Attention in a compatible runtime.
• q8_0 K/V types set cache precision separately from weight quantization.
• -np 1 configures one server slot, not a multi-user throughput result. [4][6]

GPU memory and system RAM are not a single interchangeable pool. Actual fit depends on tensor placement, cache, work buffers, operating-system use and paging behavior. The post does not provide enough memory telemetry for me to guarantee that the same command will perform identically on any machine with those two capacity numbers.

The GGUF card also warns that a sufficiently recent runtime with the relevant hybrid-model support is required. “llama.cpp installed” is not a complete reproducibility specification: record the exact build, backend, driver and model revision. [4]

5. Configured 262K context is not a validated 262K workload

-c 262144 is a context-size setting, and the inspected model config contains max_position_embeddings: 262144. Neither demonstrates that this video processed that many tokens or maintained the same speed and quality near the limit. [2][6][9]

There is a second caution in the Empero card: the student was trained on 8,192-token examples, and the author explicitly warns that long-form and long-context behavior may degrade relative to the base. That is not a claim that every longer input fails; it is a reason not to equate an inherited/configured maximum with demonstrated reliability. [3]

Increasing context changes the runtime memory and processing workload. The architecture’s full-attention cache and recurrent state should not be estimated as though every layer were an identical conventional attention layer. Without the actual runtime allocations and workload, I am not presenting a made-up “exact VRAM at 262K” figure.

To substantiate a long-context claim, I would want actual input-token counts, output budget, memory use, time to first token, decode speed and task correctness across several context lengths. A successful launch with a large limit is only the start of that test.

6. Decode speed, prompt speed and task completion are different measurements

Decode speed is how quickly the system produces tokens after processing the input. Prefill processes the prompt. A coding agent also spends time on tool calls, tests, retries, long reasoning and context rebuilding.

For scale only: at a hypothetical constant 550 prompt tokens/s, processing 10,000 uncached input tokens would take about 18.2 seconds. That is a simple division using the reported rate, not an observed latency, and not a prediction for larger contexts or cached prompts.

The useful completion metric is closer to:

Did it produce a correct patch, within the agreed time and resource budget, and pass the relevant checks without unsafe actions?

A model emitting reasoning quickly can still spend its output budget without touching a file. The later 20-task report illustrates exactly why throughput should be evaluated alongside loop rate, output limits and task outcomes. [7]

7. What to record before buying hardware or replacing an agent

Use the original report as a candidate configuration, not a purchasing guarantee.

For a reproducible local trial, record:

• Machine: CPU model, RAM capacity/speed, GPU model/VRAM, storage and whether swapping occurred.
• Artifacts: exact repository, checkpoint revision, quantized file/hash, runtime build and chat template.
• Execution: expert offload, cache types, actual context occupancy, slots, sampling and reasoning/output limits.
• Performance: cold and warm prompt latency, prefill, decode, peak RAM/VRAM and variation across runs.
• Agent outcome: task list, tool/parser failures, valid edits, test results, loops, timeouts and total time to a correct result.

Keep the harness, acceptance tests and budgets consistent when possible. If you deliberately use different quants to compare what fits the same hardware budget, say so; do not describe it as a same-quant model-quality comparison.

Download my benchmark worksheet for your own records. It is a blank reporting aid, not a benchmark I have executed.

8. Licensing and privacy still need a boundary check

The inspected Empero model and GGUF cards declare Apache-2.0, inherited from the Qwen3.6 base. That is their stated license; it does not independently validate the training-data provenance, model quality or every dependency’s terms. [3][4][5]

Local inference can keep prompts on your own hardware, but the surrounding agent may still call external tools, telemetry or remote services. Check the full route, not just the model server.

One disclosure in Empero’s card deserves attention: it says some prompts from its free community endpoint contributed to the synthetic training data. Do not assume a hosted community endpoint has the privacy properties of a local GGUF run. This disclosure is not evidence that a locally run checkpoint uploads your prompts. [3]

If running llama-server, keep network exposure deliberate. The reviewed server docs default the bind address to loopback and offer API-key controls; do not casually publish an unauthenticated endpoint or give a local coding agent unrestricted credentials. [6]

My assessment

The useful result: CPU/GPU expert placement and quantization can make a large sparse checkpoint usable on modest hardware, and this video visibly shows approximately 50 tokens/s generation.

The necessary correction: it is an Empero community distill, the weights are not wholly resident in 12GB VRAM, a 262K setting is not a long-context validation, and the later author-reported coding-agent result is more informative than the attractive speed clip.

I would shortlist such a setup for a controlled trial, not promise that it replaces a proven coding agent. Our earlier LLMSpeed investigation made the same broader distinction between estimates and measured inference. Here the additional lesson is between measured generation speed and a correct completed software task.

Buy and configure for the work you need finished—not the fastest number in a repost.

Sources

[1] https://x.com/oluwaphilemon1/status/2115957147074957420?s=46 — User-shared summary, secondary report [2] https://x.com/DogukanUrker/status/2100346050473074805 — Original developer hardware/speed post, September 16 [3] https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill/blob/bcc2dbe2f21b213625df2dc1a5a690212373af07/README.md — Empero model card, pinned checkpoint [4] https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF/blob/b1f9d1dcc3de8aa867669b0ab919384aeeb9b8d5/README.md — Empero GGUF quantization card and requirements [5] https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/995ad96eacd98c81ed38be0c5b274b04031597b0/README.md — Official Qwen3.6 base-model architecture [6] https://github.com/ggml-org/llama.cpp/blob/0ec4027a2a0183de87c007069adddc06e665f7de/tools/server/README.md — llama.cpp current server flags and security defaults [7] https://x.com/DogukanUrker/status/2101996527702941863 — Developer September 21 agentic coding result [8] https://x.com/DogukanUrker/status/2115102453666381882 — Developer October 27 benchmark correction [9] https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill/blob/bcc2dbe2f21b213625df2dc1a5a690212373af07/config.json — Pinned model context and hybrid architecture config [10] https://github.com/ggml-org/llama.cpp/blob/0ec4027a2a0183de87c007069adddc06e665f7de/common/arg.cpp — llama.cpp CPU expert placement option semantics [11] https://huggingface.co/api/models/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF/tree/b1f9d1dcc3de8aa867669b0ab919384aeeb9b8d5?recursive=false&expand=false — Pinned GGUF file metadata and exact byte sizes [12] https://github.com/DogukanUrker/BenchKit/blob/5945b05d21b8717fed3d872652a5b44d9efb2c1e/docs/patcheval.md — Pinned PatchEval contract: historical hidden-test correction [13] https://api.github.com/repos/DogukanUrker/BenchKit/git/trees/5945b05d21b8717fed3d872652a5b44d9efb2c1e?recursive=1 — Complete pinned BenchKit tree: historical artifact search boundary [14] https://github.com/DogukanUrker/BenchKit/blob/5945b05d21b8717fed3d872652a5b44d9efb2c1e/.gitignore — BenchKit local results are ignored, not committed benchmark records