# Turbo vs Stock — what the Turbo system buys on the B70 (AgentWorld + Ornith)

Head-to-head between the **Turbo** serving stack and a **stock** baseline, same weights, same GPU, same
SYCL compiler/flags — so the delta is attributable to the Turbo system alone.

| | Turbo | Stock (baseline) |
|---|---|---|
| build | `llama.cpp-sycl-moe-ready` — decode-fusion (topk-MoE router fusion +2.7%, gate-glue fusion +1.25%, single-token expert-aggregate) | `llama.cpp-sycl-moe-clean` @ `885e3e325` — same base **minus** those 3 fusion commits |
| runtime | `GGML_SYCL_DISABLE_DNN=1`, `-b 8192 -ub 4096`, **q8_0** KV | upstream defaults: DNN **on**, `-b 2048 -ub 512`, **f16** KV |
| shared | Q5_K_M weights · Intel Arc Pro B70 · icpx 2026.0, `GGML_SYCL/NATIVE/F16/GRAPH=ON` · `-ngl 99 -fa on` · llama-server + `concurrent_agents.py` |

Both models are `qwen3_5_moe` (35B / ~3B active). Runs: Turbo 2026-06-30, Stock 2026-06-30/07-01, `:8092`.
Data under `results/live_evals_stock/{agentworld,ornith}/np_pareto.md` and `.../games/`.

## 1. Throughput / agent-pareto — aggregate decode t/s (tool class, `-c 131072`)

Stock swept at np 1/2/4/8/16/24/32 (the pareto); Turbo at the matched points 1/8/16/32. (Turbo pareto here
used the bench driver's q8_0 KV; prompts are ≤775 tok so KV dtype is ~negligible — shipped f16 would read
marginally higher. The KV dtype matters only at depth — see §4.)

**AgentWorld**

| agents | stock | Turbo | Turbo/stock |
|---:|---:|---:|:--:|
| 1  | 62.31 | 62.59 | **1.00×** |
| 2  | 67.94 | — | |
| 4  | 95.68 | — | |
| 8  | 125.12 | 141.49 | 1.13× |
| 16 | 146.78 | 163.35 | 1.11× |
| 24 | 168.97 | — | |
| 32 | 186.85 | 223.95 | **1.20×** |
| structured×32 | 134.38 | 169.44 | 1.26× |
| novel×32 | 110.88 | 138.35 | 1.25× |

**Ornith**

| agents | stock | Turbo | Turbo/stock |
|---:|---:|---:|:--:|
| 1  | 62.54 | 60.87 | **0.97×** |
| 2  | 70.84 | — | |
| 4  | 99.34 | — | |
| 8  | 118.30 | 140.58 | 1.19× |
| 16 | 137.27 | 161.26 | 1.17× |
| 24 | 173.97 | — | |
| 32 | 183.17 | 221.73 | **1.21×** |
| structured×32 | 134.48 | 165.90 | 1.23× |
| novel×32 | 102.98 | 133.55 | 1.30× |

**Reading:**
- **Single-stream (np=1): parity.** 1.00× / 0.97× — within run-noise. The decode-fusion + DNN-disable + q8_0
  KV give essentially **no single-token decode gain** on this arch/quant. (Consistent with the standing
  caveat that the model-card single-stream fusion numbers were overstated.)
- **Fleet concurrency (np≥8): Turbo wins ~1.1–1.3×.** The gain grows with batch size and prompt length,
  peaking at **1.20–1.30× at 32 agents**. Turbo's value on this axis is **batched/fleet serving**, not
  single-user latency.
- **Pareto shape is identical** (both models, both builds): aggregate keeps climbing through np=32; per-agent
  decode roughly halves by np=8 (the usable "knee" — beyond it you buy total throughput at rising per-agent
  cost). Stock and Turbo trace the same curve, Turbo shifted up ~15–20% from np=8 on.

## 2. Teamwork games — parity/quality check, NOT a speed metric

| model · game | Turbo (rounds / wall s / valid / bytes) | Stock (rounds / wall s / valid / bytes) |
|---|---|---|
| AgentWorld frogger | 1 / 1163 / 6-6 / 40,439 | 2 / 1742 / 6-6 / 27,520 |
| AgentWorld maze | 1 / 958 / 6-6 / 27,483 | 2 / 1966 / 5-6 / 29,933 |
| Ornith frogger | 2 / 1374 / 6-6 / 20,155 | 2 / 1015 / 6-6 / 17,891 |
| Ornith maze | 1 / 1179 / 6-6 / 29,001 | 1 / 749 / 6-6 / 19,692 † |

† stock Ornith maze final build **fails jsdom at runtime** (`Cannot read properties of undefined (reading '0')`).

**Round wall-time is dominated by how many tokens the model chose to emit** (a bigger build = a longer
round), not by engine speed — e.g. stock's Ornith games ran *faster* only because they emitted *smaller*
builds (17.9 KB/1015 s vs Turbo 20.2 KB/1374 s). So the games are a **build-validity parity check**, not a
throughput measurement: both engines land ~6/6-valid builds within the budget (one stock build broke, one
Turbo build was unwinnable last run — build quality is stochastic per run, independent of engine). **Use §1
for speed.**

## 3. What this comparison does NOT capture (Turbo's other two documented wins)

This bench uses short prompts (125–775 tok) at fixed `c=131072`, so it exercises **batched decode only**. Two
real Turbo advantages are therefore untested here:
- **Large-prefill.** `-ub 4096` + `DISABLE_DNN` is a prefill win (~+73% at pp4096 in prior measurement); a
  125-token prefill can't show it.
- **Long-context capacity.** q8_0 KV halves KV memory → Turbo reaches `c=262144`; stock f16 KV costs 2× and
  caps lower under concurrency. Fixed 131072 here doesn't probe it.

→ **Measured in §4** (single-stream prefill + decode-vs-context, both builds).

## 4. Single-stream depth sweep — prefill + decode vs context (AgentWorld, `-c 262144 -np 1`)

The §1 pareto uses short prompts, so it can't see prefill scaling or long-context decode. This sweep pads the
prompt to real sizes and measures **three** configs (KV dtype called out because it turns out to dominate):
- **stock** — moe-clean · f16 KV · default `-ub 512` · DNN on
- **Turbo (shipped)** — moe-ready · **f16 KV** · `-ub 4096` · DNN off — *AgentWorld's actual `agentworld-serve.sh`*
- **Turbo-q8_0** — moe-ready · **q8_0 KV** · `-ub 4096` · DNN off — *the bench driver's KV, and what some other Turbo products ship*

### Prefill throughput (t/s) vs prompt size
| prompt tok | stock | Turbo (f16) | Turbo-q8_0 | **Turbo/stock** |
|---:|---:|---:|---:|:--:|
| 805 | 635 | 832 | 768 | 1.31× |
| 3,313 | 856 | 1522 | 1538 | 1.78× |
| 6,963 | 849 | 1577 | 1571 | **1.86×** |
| 14,563 | 813 | 1452 | 1461 | 1.79× |
| 29,713 | 712 | 1171 | 1180 | 1.64× |
| 61,341 | 561 | 818 | 820 | 1.46× |
| 129,325 | 381 | 487 | 486 | 1.28× |

Prefill is KV-insensitive (f16 ≈ q8_0) — the win is `-ub 4096` + DNN-off. **Turbo prefills 1.3–1.86× faster**,
peaking ~1.86× around 7k tokens, tapering to 1.28× at 129k (deep prefill is increasingly attention-over-KV
bound, which both pay).

### Decode throughput (t/s) vs context depth
| context tok | stock (f16) | **Turbo (f16, shipped)** | Turbo-q8_0 | **shipped/stock** | q8_0-KV cost |
|---:|---:|---:|---:|:--:|:--:|
| 805 | 80.3 | 91.7 | 84.9 | 1.14× | 1.08× |
| 3,313 | 78.9 | 89.3 | 74.1 | 1.13× | 1.20× |
| 6,963 | 76.5 | 86.4 | 62.7 | 1.13× | 1.38× |
| 14,563 | 72.0 | 81.1 | 47.0 | 1.13× | 1.73× |
| 29,713 | 65.5 | 73.3 | 32.1 | 1.12× | 2.29× |
| 61,341 | 55.3 | 60.6 | 19.3 | 1.10× | 3.14× |
| 129,325 | 41.3 | 44.0 | 10.4 | 1.07× | **4.23×** |

- **Shipped Turbo (f16) decodes 1.07–1.14× faster than stock at every depth** — the fusion build helps decode
  and holds with context. There is **no decode penalty** (an earlier draft compared the q8_0-bench config and
  wrongly showed one).
- **q8_0 KV is the villain, not the build.** Turbo-q8_0 decode collapses with context — 1.4× slower than
  Turbo-f16 at 7k, **4.2× slower at 129k**. The B70 SYCL **q8_0-KV flash-attention path is badly suboptimal at
  depth** (dequant cost scales with KV length). AgentWorld ships **f16** (unaffected); **any Turbo product
  serving q8_0 KV should switch to f16** unless it genuinely needs the KV-memory halving for context capacity.

## Bottom line
Measured, un-inflated, for the **shipped f16 config**:
- **Prefill: 1.3–1.86× faster** than stock (peak ~1.86× at ~7k prompt) — the `-ub 4096` + DNN-off win.
- **Single-stream decode: 1.07–1.14× faster** than stock at **all** depths (0.8k→129k) — the fusion build, no
  penalty. (Not "parity" — an earlier note said that; it was an artifact of benching q8_0 KV.)
- **Fleet decode: ~1.2× aggregate** at 32 concurrent agents (1.1–1.3× across np 8–32).
- **Zero quality cost** — same weights, lossless decode, games validity parity.
- **Bonus finding for the product line:** q8_0 KV (bench driver + some other Turbo builds) is up to **4.2×
  slower decode at 129k** on this SYCL backend → move those to f16 KV, or fix the q8_0 FA kernel.

So the Turbo system is a real win on every axis for the shipped config; the only "loss" anywhere was a KV
dtype (q8_0) that AgentWorld doesn't use — and it's a lever worth pulling for the products that do.

## Reproduce
```bash
# stock baseline (moe-clean, upstream-default config)
bash /home/frosty40/qworld_turbo/bench/run_stock_evals.sh <gguf> <alias>-stock <tag> 8092
# turbo (moe-ready, ship config)
bash /home/frosty40/qworld_turbo/bench/run_live_evals.sh  <gguf> <alias>-turbo <tag> 8092
```