Reference — QB2 vs GPU Hardware
Known-good inference throughput for common models on datacenter and consumer GPUs (fp16/bf16, batch 1, from MLPerf, vendor docs, and community benchmarks). QB2 numbers are from this bestiary at fp32. Methodology differs — treat as directional, not apples-to-apples.
| Model | Task | QB2 Blackhole | A100 80GB | RTX 4090 | RTX 3090 | M2 Max | Notes |
|---|---|---|---|---|---|---|---|
| GPT-2 (124M) | text-gen | 539 tok/s (forge) | 3,000–4,000 tok/s | 1,500–2,000 | 800–1,200 | 200–400 | QB2 at fp32 · GPUs at fp16 |
| GPT-2 (124M) | text-gen (XLA) | 430 tok/s (xla) | — | — | — | — | JAX/PJRT, single chip |
| BLOOM-1.1B | text-gen | 3,856 tok/s (xla 4-chip) | 400–600 tok/s | 200–350 | 100–200 | 50–100 | QB2: 4-chip data-parallel batch=4 · GPUs: single device batch=1 |
| AlexNet | image-cls | 169 ms/smp | ~2 ms | ~3 ms | ~4 ms | ~10 ms | QB2 at fp32; GPUs highly optimized for AlexNet |
| MobileNetV2 | image-cls | 484 ms/smp | ~3 ms | ~4 ms | ~6 ms | ~15 ms | Depthwise conv not yet optimized on Blackhole |
| BEiT (XLA) | image-cls | 23,922 tok/s (xla) | — | — | — | — | Token = image patch; JAX/PJRT first compile |
GPU reference sources: NVIDIA MLPerf v3.1 (A100), community benchmarks (RTX 4090/3090), Apple ML Research (M2 Max). TT Blackhole is a new architecture — some ops (depthwise conv, certain attention patterns) are not yet fully tuned, which explains cases where QB2 is slower at batch=1. Throughput at larger batch sizes and with metal-optimized kernels will differ.
Chip Leaderboard
Points scored per chip. Each first-ever compile earns rarity-weighted points; repeat successes on seed models add bonus points.
Compiled Models
Every model successfully compiled and executed on TT hardware. Click column headers to sort. First Voice is the model's real inference output decoded from raw logits.
| Model | Rarity | Backend | Task | Chips | Params | Compile | TTFT | Prefill | Decode | Runs | Run # | Chip |
|---|