A roguelike model-compilation game for Tenstorrent Blackhole hardware.
Discovers AI models from the wild, compiles them, scores by rarity,
and builds a bestiary of everything the silicon has ever learned to run.
Live recording · 56 seconds · 4 chips · BLACKHOLE · forge ·
regenerate with
bash scripts/record_demo.sh
The Story
Tenstorrent ships real silicon — Blackhole chips with enough compute to
run the entire modern transformer zoo — but testing which models actually
compile requires patience, iteration, and a lot of terminal windows.
Compiletron started as the answer to one question:
"what if the compilation loop were a game?"
Origin
compiletron.py — The First Draft
A single 300-line script. Four tmux panes. A hard-coded list of
101 models from tt-forge-models. Each pane ran
forge.compile() in a loop and printed pass/fail.
No scoring, no discovery, no state between runs. Blunt and effective.
The Upgrade
expedition.py — Live Discovery + Roguelike Scoring
The static list became a live HuggingFace frontier scanner:
every run discovers newly-published models, filters by download count and
quality signals, pre-fetches weights in parallel, then dispatches one model
per chip across all available hardware. A roguelike scoring system
replaced the binary pass/fail — rarity, newness, streaks, and
First Voice (post-compile inference) all add points.
A persistent Bestiary records every model ever compiled.
The Interface
Textual TUI — Setup → Run → Summary
The tmux four-pane layout became a full
Textual TUI with three screens:
Setup (configure chips, backend, filters),
Run (live chip panels, expedition log, streaming worker output),
and Field Report (NATO operative codenames ALPHA/BRAVO/CHARLIE/DELTA
ranked by points, mission classification badge, NEW INTELLIGENCE section for
first-ever models with INTERCEPT labels for first-voice text, TARGETS AT LARGE
for failures). Every run ends like a declassified debrief.
Dual Backend
tt-forge + XLA/PJRT — Two Stacks, One Expedition
With the pjrt-plugin-tt landing in the Tenstorrent
software stack, JAX/Flax models could now compile directly onto
Blackhole via the PJRT interface. The XLA backend joined as a
first-class citizen: per-chip backend selection, automatic routing
(auto mode sends JAX models to XLA and PyTorch to forge),
and a mixed mode for side-by-side backend comparison
on the same run.
Two Compilation Stacks
Compiletron runs both of Tenstorrent's compilation pathways simultaneously.
The --backend auto mode routes each model to the right stack
based on its framework; mixed mode splits chips evenly for
direct comparison.
BACKEND A
tt-forge (PyTorch)
Compiles PyTorch nn.Module graphs via forge.compile()
Covers the full tt-forge-models seed zoo (230+ models)
Output: compiled TorchScript-style graph on TT device
Requires: source ~/tt-forge-fe/env/activate
BACKEND B
XLA / JAX (PJRT)
Compiles Flax/JAX models via jax.jit + PJRT plugin
Uses pjrt-plugin-tt — the TT PJRT backend for XLA
Covers JAX loaders in tt-forge-models and HF Flax models
JIT compilation on first call — XLA traces, then dispatches to TT
Requires: ~/tt-xla/venv with pjrt-plugin-tt
From tt-forge to XLA: The PJRT Story
tt-forge (formerly TT-Buda) was the first-party
compilation framework: take a PyTorch model, lower it through tt-forge's
graph passes, and produce a runnable binary for TT hardware. It works, it's
battle-tested, and it handles the vast majority of the PyTorch ecosystem.
XLA + PJRT opened a second door. The PJRT (Plugin Registration
and Targeting) interface is the standard plugin contract that XLA-based
frameworks (JAX, TensorFlow) use to target hardware backends. Once
pjrt-plugin-tt landed, the entire JAX/Flax model ecosystem
became compilable on Tenstorrent hardware — including the Flax variants of
BERT, GPT-2, T5, and AlexNet that live in tt-forge-models.
Three compatibility patches are applied automatically at worker start
to handle the pjrt + JAX 0.7.x + Flax 0.8.x interaction.
# Select backend at runtime — or let the router decide$python3 expedition.py run --tui --backend forge # PyTorch only$python3 expedition.py run --tui --backend xla # JAX/PJRT only$python3 expedition.py run --tui --backend mixed # even=forge, odd=xla$python3 expedition.py run --tui # auto-route per model
The Roguelike Layer
Every model that compiles is a victory. Not all victories are equal.
Compiletron borrows from the vocabulary of roguelikes — dungeon crawlers
like Nethack, Spelunky, and Caves of Qud —
to make each run feel like an expedition into the unknown.
★ Rarity Tiers
Each model is classified by HuggingFace download count into a rarity
tier. Obscure models that hardly anyone has tried are Legendary.
Mainstream models are Common. Compiling a Legendary is
worth 5× the base points.
★★★ Legendary★★ Epic★ Rare◇ Uncommon· Common
◈ Newness Bonus
Models published in the last 24 hours are Zero-Day
(+300 pts). Last week is Hot (+100 pts).
Last month is Fresh (+50 pts).
Compiling something the day it was published is the highest honour.
🔥 Streak Multiplier
Each chip earns a streak for consecutive successful compiles. Streaks
unlock bonus points and reset on any failure — a measure of how far
the hardware has run without interruption.
🗣 First Voice
After a successful compile, the worker runs a themed inference pass —
feeding a curated prompt from lib/expedition/sampler.py
(short fiction, images, questions). The decoder in
decoder.py turns raw logits into readable text.
First Voice earns +100 pts and a journal entry.
Scoring Reference
Event
Points
Notes
Base success
+200
Any successful compile
First-ever compiled
×5 bonus
Not previously in bestiary → ×5 multiplier on all points
Legendary rarity
×2.0
< 1,000 downloads on HuggingFace
Epic rarity
×1.75
1K–10K downloads
Rare rarity
×1.5
10K–100K downloads
Uncommon rarity
×1.2
100K–1M downloads
Zero-day newness
+300
Published < 24h ago
Hot newness
+100
Published < 7 days ago
Streak (🔥×N)
+50×N
Consecutive successes on same chip
First Voice
+100
Inference produces meaningful output
Failure penalty
−10
Compile or inference error
The Bestiary
data/bestiary.json is the permanent record: every model that
has ever compiled on this hardware, with its artifact shape, compile time,
First Voice text, chip, and run number. It accumulates across runs and is
never overwritten. The chip history inside tracks all-time compilation
totals per chip — a running log of what each device has proven it can run.
Per-run journals are written to data/expeditions/run_NNN.md
with prose summaries of every First Voice result — a field log of what the
hardware has spoken.
Side Quests — Chips Never Idle
Some models require the entire mesh — all four Blackhole chips running in
lock-step. While the TUI holds those chips in reserve and waits for quorum,
every other free chip would otherwise sit silent. Side quests
eliminate that dead time.
⚡ How They Trigger
When a multi-chip model enters the queue (MESH ASSEMBLING),
any chip that finishes its main-queue work is immediately dispatched a
side quest — a fast, curated seed model that compiles in under 20 seconds.
No chip idles while the RALLY assembles.
✍ Tracked Separately
Side quest results carry is_sq=True in RunState. They earn
real points and appear as a ⚡ BONUS HAUL line in
the Field Report — distinguished from main-queue results so leaderboard
scores reflect only the primary expedition.
✓ Automatic Dedup
Before each side quest dispatch, the controller scans every chip's
in-flight and completed model list. Any model already run or currently
compiling on any chip is skipped, so every side quest result within a
run is unique.
▶ RALLY Interrupt
When enough chips free up to form the RALLY quorum, a
_rally_interrupt_flag fires. No new side quests are
launched, but in-flight ones run to natural completion before the
multi-chip compile begins.
Side Quest Pool
The pool is a hand-curated set of fast, reliable forge models — all
single-chip, all proven to compile in under 30 seconds. They are dispatched
in order, skipping any model already seen this run.
Model
Task
Rarity
Typical compile
MobileNetV2
image-classification
common
~10 s
GhostNet
image-classification
uncommon
~20 s
GoogLeNet
image-classification
common
~10 s
EfficientNet-Lite
image-classification
uncommon
~12 s
DenseNet-121
image-classification
uncommon
~15 s
ResNet
image-classification
common
~12 s
SqueezeBERT
text-classification
rare
~18 s
DeiT
image-classification
uncommon
~10 s
In the Field Report
After the run, side quest results appear as a compact bonus summary in the
summary screen, separate from the main OPERATIVE STATUS leaderboard:
Real results from actual expeditions on Tenstorrent Blackhole hardware.
Every entry was compiled live — weights fetched, graph lowered through tt-forge,
binary uploaded to the chip. First Voice output is the model's real
inference on a themed prompt, decoded from the raw logit tensor by
decoder.py.
View full leaderboard →
Some models fought back. These are the ones that escaped.
✗ Failed
Yosua69/blt-moe-distilled
unsupported_arch5 attempts
forge: unsupported architecture in MoE routing layer
✗ Failed
bi_lstm_crf/pytorch
seedforge_internal2 attempts
forge: graph lowering assertion failed
Performance Benchmarks — Tenstorrent QB2
Measured on a Tenstorrent QB2 (4× Blackhole) during Expeditions #067–#068 using
--bench-passes 5: two warm-up passes followed by five timed inference
passes per model. Compile time is wall-clock for forge.compile().
p50 infer is the median of the five timed passes.
Throughput is tokens/sec for language models and
ms/sample for vision/embedding models.
Verified 5-pass bench results from the community on their own Tenstorrent
hardware. Each entry was reviewed and merged by a maintainer.
Submit your results →
Loading community data…
Architecture
Compiletron is a thin orchestration layer over two independent compilation
stacks. The TUI drives everything; workers are subprocesses that stream
results back through CSV files.
Each run scans the tt-forge-models zoo (242 loaders after
filtering ONNX and IRD-locked models) and the HuggingFace Hub for
recently-created models. One model per author-family per run.
Seed models already in the bestiary are skipped unless
--staples forces a re-run.
Per-Chip Workers
Each chip gets its own subprocess — expedition_worker.py
(forge) or expedition_worker_xla.py (XLA) — launched by
the TUI with its device assignment. Results stream back through
/tmp/expedition_results_chipN.csv. Chips never share state.
Quick Start
# Clone and install$git clone git@github.com:tsingletaryTT/tt-forge-compiletron.git$cd tt-forge-compiletron$pip install -r requirements.txt# Activate the tt-forge environment (required for forge backend)$source ~/tt-forge-fe/env/activate# Launch the TUI — auto-detects chips, discovers live HF models$python3 expedition.py run --tui# Seed-only run: 4 chips, 1 model per chip, no HF download$python3 expedition.py run --tui --seed-only --limit 4 --chips 4 --no-predownload# XLA backend (JAX/PJRT) — requires ~/tt-xla/venv$python3 expedition.py run --tui --backend xla --seed-only# Mixed mode: even chips = forge, odd chips = XLA$python3 expedition.py run --tui --backend mixed# Staples: re-run proven seed models (regression test after forge update)$python3 expedition.py run --tui --staples --limit 8# High-quality frontier: min 1000 downloads, max 7B params$python3 expedition.py run --tui --min-downloads 1000 --max-model-params 7# Benchmark mode: 5 timed inference passes per model (p50/p95 stats)$python3 expedition.py run --tui --bench-passes 5# With shape sweep (varies seq_len for LLMs, resolution for vision)$python3 expedition.py run --tui --bench-passes 5 --bench-shapes# View bench results from the last run$python3 scripts/show_perf_stats.py
XLA Environment Setup
# The XLA worker uses ~/tt-xla/venv — set it up once:$git clone https://github.com/tenstorrent/tt-xla ~/tt-xla$python3.12 -m venv ~/tt-xla/venv$cd ~/tt-xla && pip install -e python_package/pjrt_plugin_tt$~/tt-xla/venv/bin/pip install jax jaxlib flax transformers
Record a Demo
# Capture a live expedition run as an asciinema cast$bash scripts/record_demo.sh# Capture a bench run (5 passes per model) — stats shown at end$bash scripts/record_demo.sh --bench# Outputs docs/demo.cast — embed with the asciinema player above# Or regenerate the scripted placeholder (no hardware required)$python3 scripts/gen_demo_cast.py > docs/demo.cast