Benchmarks & token stats

Ornith-1.0-35B — agent-concurrency throughput Pareto

Batched generation throughput as the number of concurrent agents grows (llama.cpp, GGUF-NVFP4). Aggregate generation reaches 372.28 t/s at 256 agents; the separate total-throughput column reaches 848.67 t/s. Per-agent speed and single-reply latency degrade as concurrency rises. The highlighted 32-agent row is the recorded fleet selection.

AEON QWEN — concurrency Pareto

NVFP4 MTP-XS on vLLM with z-lab DFlash speculative decode. DFlash makes single streams fast (~46–48 t/s), so aggregate plateaus early — the knee is 8 (91% of peak at 2× the per-agent speed of 16). The highlighted 8-agent row is its build-challenge fleet.

conc=1 is cold-start (first request compiles CUDA graphs); true single-stream is the rate at conc 2–4.

Ornith-AEON — concurrency Pareto

Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4 on the local KV-fix vLLM image with z-lab DFlash drafter n=6. This is measured through the OpenAI-compatible API using real usage token counts and batch wall-clock time; the selected build fleet is the capped 16-agent serve. The highlighted 16-agent row is its build-challenge fleet.

Qwen3.6-35B-A3B base — concurrency Pareto

Generic Qwen3.6-35B-A3B UD-Q6_K GGUF on Spark/GB10 via llama.cpp-qwen36, -c 32768 -np 32, q8 KV, FlashAttention, and fixed 256 generated tokens per agent using min_tokens=256 + ignore_eos. This is a GGUF llama.cpp condition, not the vLLM/NVFP4 stack. The highlighted 32-agent row is its build-challenge fleet.

Qwen3.6-35B-A3B Q5 Turbo on Intel Arc Pro B70 — concurrency Pareto

Qwen3.6-35B-A3B-UD-Q5_K_XL GGUF served by local Turbo/llama.cpp SYCL on Intel Arc Pro B70 with a 12-slot unified KV configuration and 262144 context. This live OpenAI-compatible run peaks at 158.5 decode tok/s at the 12-agent serve ceiling, with 16.3 tok/s per agent. The highlighted 12-agent row is the 12-slot B70 live serve ceiling.

Qwen AgentWorld-35B-A3B Spark — concurrency Pareto

Qwen/Qwen-AgentWorld-35B-A3B local GGUF_NVFP4.official artifact on Spark/GB10 via llama.cpp-qwen36, -c 32768 -np 32, q8 KV, FlashAttention, and fixed 256 generated tokens per agent using min_tokens=256 + ignore_eos. This is the controlled Spark rerun of AgentWorld, kept separate from the older site AgentWorld CSV. The highlighted 32-agent row is its build-challenge fleet.

Qwen3-Coder-30B-A3B-Instruct BF16 — concurrency Pareto

Official Qwen/Qwen3-Coder-30B-A3B-Instruct BF16 safetensors on Spark/GB10 via vLLM 0.23.0, max_model_len=32768, max_num_seqs=32, FlashAttention, and unquantized FlashInfer CUTLASS MoE. No REAP, no DFlash, no drafter, no quantized derivative. Fixed 256 generated tokens per agent using min_tokens=256 + ignore_eos. The highlighted 32-agent row is its build-challenge fleet.

NVIDIA Nemotron-3-Super-120B-A12B-NVFP4 — concurrency Pareto

Official NVIDIA NVFP4 safetensors on Spark/GB10 via vLLM 0.23.0, max_model_len=8192, max_num_seqs=1, kv_cache_memory_bytes=4G, and eager execution. This is a single-agent scout condition, not a wide concurrency Pareto, because this Spark serve was constrained to one active sequence. The highlighted 1-agent row is its build-challenge fleet.

NVIDIA Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 — concurrency Pareto

Official NVIDIA Puzzle-75B-A9B NVFP4 checkpoint on Spark/GB10 via vLLM 0.23.0, max_model_len=8192, MTP speculative decode num_speculative_tokens=1, max_num_seqs=32, kv_cache_memory_bytes=16G, FlashInfer attention/Mamba, ModelOpt mixed NVFP4/FP8, and eager execution. Fixed-token sweep used min_tokens=192 + ignore_eos; the selected build fleet is the measured 32-agent ceiling, while game builds used the separate context-calibrated 1-agent team. The highlighted 32-agent row is its build-challenge fleet.

AgentWorld-35B-A3B — concurrency Pareto

GGUF-NVFP4 (qwen35moe) on llama.cpp at -np 32 — same architecture and size as Ornith, and its curve tracks Ornith's, so the knee sits at 32. The highlighted 32-agent row is its build-challenge fleet.

Gemma-4-26B-A4B — concurrency Pareto

NVFP4 weights with z-lab DFlash speculative decode on vLLM — single-stream 227 t/s (DFlash), aggregate peaks 807 t/s at 64 agents. The knee sits at 32 (96% of peak at 2× the per-agent speed of 64). This is the model used for the 20-min teamwork builds. The highlighted 32-agent row is its build-challenge fleet.

Gemma-4-26B-A4B NVFP4 current plain vLLM — concurrency Pareto

Current live gemma4 plain vLLM serve on Spark/GB10, max_model_len=32768, max_num_seqs=48, no DFlash/speculative decode. The 48-agent row is the measured server ceiling and the aggregate decode peak in this sweep; game builds use the separate 5-agent context-calibrated team size. The highlighted 48-agent row is its build-challenge fleet.

Qwen3.6-35B-A3B NVFP4 — concurrency Pareto

NVIDIA nvidia/Qwen3.6-35B-A3B-NVFP4 on Spark/GB10 via vLLM 0.23.0, modelopt NVFP4 (W4A16) + FP8 attention, MARLIN MoE, max_model_len=32768. This is a 4-bit quantized model — roughly 3× smaller in VRAM than the BF16 base, so more KV-cache budget remains and aggregate decode is 660 t/s at only 32 agents. The sweep stops at 32 because the Pareto knee is there; higher concurrency only degrades per-agent speed (20.6 t/s → lower), it does not raise aggregate further on a single GB10. The highlighted 32-agent row is its build-challenge fleet.

InternScience Agents-A1 BF16 — concurrency Pareto

InternScience/Agents-A1 — 35B MoE agentic model, BF16 safetensors on Spark/GB10 via vLLM 0.23.0, FlashInfer CUTLASS MoE, max_model_len=131072. Full-precision (BF16) means a 65.5 GiB model footprint, leaving a smaller KV-cache than the NVFP4 condition, so aggregate is 349 t/s at 32 agents — within 1.4% of the Qwen3-Coder-30B-A3B BF16 baseline (344 t/s), as expected for the same architecture class. The sweep stops at 32 (Pareto knee); the GPU is saturated beyond that. The highlighted 32-agent row is its build-challenge fleet.

Intel Arc Pro B70 — cross-model tables

Single B70, llama.cpp SYCL, same-architecture Qwen3.5-35B-A3B fine-tunes plus DREAM-v0-7B. Date: 2026-06-27.

B70 figures

Concurrency Pareto — aggregate decode tok/s

Peak is 48 agents; 56 agents is the 30 GiB memory cliff. Production knee is 32 agents.

B70 live server fleet — tool prompt aggregate decode tok/s

Live values are simultaneous OpenAI-style requests through llama-server; Quest is qwen3moe and uses a 131k safe context ceiling.

Context Pareto — aggregate decode tok/s

Concurrency and per-agent context trade against the same B70 memory budget.

Quest Turbo live context ceiling

DREAM diffusion Pareto

B70 game-build comparison

Raw B70 report: CROSS_MODEL_B70.md · CROSS_MODEL_LIVE_B70.md · ORNITH_LIVE_EVALS.md · NEX2_LIVE_EVALS.md · SIQ_LIVE_EVALS.md · QUEST_LIVE_EVALS.md · LAGUNA_XS21_LIVE_EVALS.md.

Downloads

results_throughput.csv · results_throughput_aeon.csv · results_throughput_aeon_ornith.csv · results_throughput_qwen36_35b.csv · results_throughput_qwen36_35b_q5_turbo.csv · results_throughput_agentworld_spark.csv · results_throughput_qwen3_coder_30b_a3b_instruct_bf16.csv · results_throughput_nemotron3_super.csv · results_throughput_nemotron_labs_3_puzzle_75b_a9b_nvfp4.csv · results_throughput_agentworld.csv · results_throughput_gemma.csv · results_throughput_gemma4_nvfp4_current.csv · results_throughput_qwen36_35b_nvfp4.csv · results_throughput_agents_a1.csv