# Cross-Model LIVE Study on the Intel Arc Pro B70 — fleet bench + teamwork games

Companion to `CROSS_MODEL_B70.md` (offline batched-bench Paretos). This is the **live-measured** layer
of the 2026-06-29 campaign: real concurrent agent requests through `llama-server`, teamwork build games,
and per-game code-quality scoring. Adds **Quest Turbo (qwen3moe)** as a 5th model on a different arch.

All on a single **Intel Arc Pro B70** (30.3 GiB), llama.cpp SYCL, Q5_K_M, ship config
`-ngl 99 -fa on -ctk/-ctv q8_0 -b 8192 -ub 4096`, `GGML_SYCL_DISABLE_DNN=1`, `--jinja`.

## Models

| Model | Arch | Active | Context ceiling (this build) | Notes |
|---|---|---|---:|---|
| AgentWorld-35B-A3B | `qwen3_5_moe` | ~3B | 262144 | reference (fully benched earlier) |
| Ornith-1.0-35B | `qwen3_5_moe` | ~3B | 262144 | |
| NEX2 (Nex-N2-mini) | `qwen3_5_moe` | ~3B | 262144 | |
| SIQ-1-35B | `qwen3_5_moe` | ~3B | 262144 | reasoning finetune |
| **Quest-30B-RL** | **`qwen3moe`** | ~3B | **131072** | new arch; RL reasoning model |

## 1. Live agent-fleet concurrency (aggregate decode t/s, tool class, `-c 131072`)

Real simultaneous requests through the server (`concurrent_agents.py`). See
`cross_charts/01_live_concurrency.png`.

| agents | AgentWorld | Ornith | NEX2 | SIQ | **Quest (qwen3moe)** |
|---:|---:|---:|---:|---:|---:|
| 1 | 62.9 | 63.1 | 62.7 | 63.1 | **66.5** |
| 8 | 141.4 | 142.2 | 142.3 | 141.9 | **164.7** |
| 16 | 163.0 | 163.4 | 163.0 | 162.8 | **223.0** |
| 32 | 223.5 | 224.9 | 224.3 | 223.7 | **335.5** |

- **The four `qwen3_5_moe` models are statistically identical** (same topology + quant + serve) — within
  run noise at every concurrency. Finetuning (reasoning, RL, agent-aug) does **not** move raw decode t/s.
- **Quest (`qwen3moe`) is the fastest fleet model — 335 t/s aggregate at 32 agents (+50% vs the family).**
  Its lighter per-token cost batches harder, and per-agent stays higher (11.3 vs ~7.9 t/s at 32).

## 2. The Quest tradeoff: wide-shallow vs deep-context

Quest wins concurrency but pays for it at depth. Single-agent decode vs context (live, `-c 131072`):

| context (prompt tok) | Quest decode t/s |
|---:|---:|
| 805 | 73.2 |
| 14,563 | 19.5 |
| 61,341 | 5.6 |
| 129,325 | 2.8 |

A ~26× collapse from shallow→129k. The `qwen3_5_moe` family's gated-delta-net instead holds context to
**262144** at steadier rates (e.g. ~22 t/s aggregate at 65k×3 agents — see `CROSS_MODEL_B70.md` §4).
**Quest also can't exceed c=131072 on this build** (the server hangs at 262144). Plus, **`llama-batched-bench`
segfaults on `qwen3moe`** (rc=139, even at tiny ctx), so Quest's Paretos had to be server-measured.
→ **Quest for wide, shallow agent fleets; the 35B family for deep single-agent context.**

## 3. Teamwork build games — code quality (Claude Opus 4.8 judge, /50)

Canonical `agentic-arcade/teamwork` harness (team 6 / merge-top 2 / 15-min, whole-game workers + merge +
never-regress jsdom gate). See `cross_charts/02_code_quality.png`.

| Model | Frogger /50 | Maze /50 | Frogger verdict | Maze verdict |
|---|:--:|:--:|---|---|
| **AgentWorld-35B** | **42** | **41** | winnable, polished | **only working maze in the study** |
| SIQ-1-35B | **42** | 30 | authentic 5-home, winnable | non-functional (tile/px units → frozen player) |
| NEX2-mini | 37 | 31 | winnable, no river drown | invisible + unshootable (radius undefined) |
| Ornith-1.0 | 33 | 34 | unwinnable (`levelComplete` never called) | can't shoot (radius undefined) |
| Quest-30B | **FAIL** | 29 | no build (runaway reasoning, empty content) | best engine, unplayable (overlay never hides) |

### Finding A — every campaign model breaks the maze; only the reference doesn't
AgentWorld's teamwork maze (41) is the **only working maze**. All four models under test shipped a maze
that **loads, validates (jsdom), and passes the playability assertions — but is not playable**, each via a
different mechanism:
- **Ornith / NEX2:** entity `radius` defined on the *type* but never assigned to the *instance* →
  `dist < a.r + b.r` is `NaN` → can't shoot / can't be hit (NEX2 also draws robots at `NaN` → invisible).
- **SIQ:** tile-vs-pixel **unit confusion** (`canMove(x,y,6)` with a 6-*tile* radius on a 13-tile map) →
  player spawns frozen, bullets spawn off-map, contact damage covers 92% of the map.
- **Quest:** correct engine, but the DOM start overlay is **never hidden** and `init()` forces
  `PLAYING`, so the game runs invisibly under a permanent "PRESS SPACE" screen (+ black-on-black walls).

The teamwork harness guarantees *runnable* and *static-gate-passing*; it does **not** guarantee *playable*.
The maze — continuous-coordinate collision + multi-system UI — is where every model trips on a **semantic**
bug no JS-error / assertion check can see. (This is exactly the gap the campaign's CODE_QUALITY_EVAL was
built to expose, and why a strong LLM source-reviewer is the right judge.)

### Finding B — frogger is the easier artifact; quality tracks model strength
Grid-locked, fewer continuous interactions → frogger holds up far better and **ranks the models**:
SIQ 42 ≈ AgentWorld 42 > NEX2 37 > Ornith 33 > Quest (failed). SIQ (reasoning finetune) produced the
campaign's most complete, genuinely-winnable frogger (authentic 5-home mechanic, 3 levels, real drowning).
Ornith's looks complete but is **unwinnable** (`levelComplete()` defined, never called) — the frogger
analogue of the maze pattern.

### Finding C — reasoning helps quality but can run away
SIQ's reasoning training produced the best frogger *and* the most ambitious (if broken) maze. Quest's RL
reasoning **ignores `enable_thinking:false`** and, on the frogger prompt, never terminates within the 32k
token budget — generating 32k reasoning tokens with **empty `content`** (a 30-min dead seed). Same
capability that lifts SIQ sinks Quest when the reasoning doesn't converge.

## 4. Bottom line

- **Speed:** Quest (`qwen3moe`) is the B70 fleet-throughput winner (335 t/s @ 32 agents); the `qwen3_5_moe`
  quartet is identical (~224) and only it reaches 262144 context.
- **Build quality:** finetunes do not change throughput but **do** change code quality. Only AgentWorld
  ships a working maze; SIQ leads on frogger; every model has at least one game with a validator-invisible
  core bug. **Static gates are necessary but not sufficient — source-level review catches what they miss.**

## Artifacts
Per-model detail + game HTML on branches `evals/{ornith,nex2,siq,quest-turbo}`. Charts:
`cross_charts/01_live_concurrency.png`, `cross_charts/02_code_quality.png`. Live data:
`benchmarks/B70/data/{ornith,nex2,siq,quest}_*.tsv`.