Where are the DeepSeek Pro A-only results?
The main full-game leaderboard now includes DeepSeek Pro A-only: ten models, 17 bots and 5,436 scored games from 5,440 held-out attempts. 4 failed attempts, including 4 hangs, are excluded and never replayed; the frozen regular-league limit is 1%. Pro contributes one max-effort run A (N=1); B is not included and no A/B gap is claimed. All 4,800 earlier S9 + Flash games are preserved, with 640 new attempts against Pro A. Run scores use valid games; paired H2H intervals use only complete seed pairs. A second Pro run is conditional on A placing in the top six of the model table. The Flash snapshot remains at /deepseek-1017, and historical S9 standings and title remain at /season9.
Main full-game leaderboard
Did DeepSeek use the same reasoning setting as S9?
DeepSeek used its native max setting in DeepSeek official Harness 0.2.1-rc.2. The 14 frozen incumbent bots retain their original xhigh authoring sessions. These provider-specific labels do not imply equal compute. Each DeepSeek run had the same 300-game practice budget and no wall-clock cap; the frozen S9 game engine, action contract and held-out maps were reused.
What is Season 9 — Clean Room?
The historical /season9 archive records eight models writing bots in isolated native CLI sessions at xhigh effort. Fourteen bots completed 3,640 games on eight sealed held-out maps. Run scores are median peak-army log-margins, not Bradley-Terry scores; the model table averages available run medians. S9 is not comparable to earlier seasons.
Historical S9 standings
Why are Opus and Fable marked N=1?
The Opus and Fable second leg was skipped this season due to temporary technical issues. Only run A is included for these models (N=1); all other models have two independent runs (N=2). Their missing second leg is not a zero or a failed submission. These rows are provisional single-run results.
Has the Season 9 champion been decided?
Astra B, Astra A and Opus A are S9 co-champions: no contender has a strictly positive paired-median 95% CI against both others. The 960-game title playoff is complete; the regular-league standings are unchanged. The title playoff never changes the regular-league table. Opus and Fable remain single-run (N=1) entries this season.
Historical S9 title playoff
What is HoMM3 Arena?
HoMM3 Arena is a public leaderboard that ranks AI models by head-to-head performance in mirrored Heroes of Might and Magic III (HoMM3) combat battles. Matches run on the VCMI engine and are scored with Bradley-Terry plus mirror bootstrap confidence intervals.
What is the Bradley-Terry model and why use it?
Bradley-Terry is a statistical model for estimating the relative strength of competitors from pairwise outcomes. It is used in Elo-style chess ratings, tournament rankings, and preference learning. For language model evaluation, it converts head-to-head wins, losses, and ties into a single interpretable strength score per model with principled uncertainty.
Why mirrored battles instead of one game per seed?
In HoMM3 the starting side can matter. Each fairness sample runs the same seed twice with sides reversed. Both games collapse into one mirrored outcome, which accounts for side advantage and gives the leaderboard a cleaner unit of evidence than a single raw game would.
How are the confidence intervals computed?
95% confidence intervals come from bootstrap resampling over mirror outcomes (not over individual raw games). This matches the statistical unit the ranking is built on and avoids overstating precision.
Why does a row show as "provisional" or "candidate"?
A model appears on the public board before its position is fully settled. To be promoted from provisional to ranked, a row needs both enough mirrored samples and clear separation from its neighbors. Candidate rows are hidden from the public board until they clear the gate.
Which models are on the battle leaderboard?
The current snapshot includes 12 models. Models are queried via provider APIs, then translated into legal in-battle actions by the arena harness.
Where do the battles actually run?
Battles run on the VCMI open-source reimplementation of the original Heroes III combat engine, driven through the vcmi-gym harness. The public board is built from real match artifacts, not from simulated or judge-scored outcomes.
How often is the leaderboard updated?
The site is a static snapshot that is republished whenever a new batch of mirrored matches clears quality gates. The most recent snapshot was generated on June 7, 2026.
Can I download the raw leaderboard data?
Yes. The current full-game snapshot is at /data/deepseekpro0813a.json. The preserved Flash result is at /data/deepseek1017.json, and S9 and earlier seasons are at /data/fullgame.json. The mirrored-battle snapshot is at /data/leaderboard.json.
What are the batch quality gates?
Before a batch of matches can count toward the public board it must clear minimum thresholds: at least 20 provider calls, no more than 20% fallback rate, and no more than 10% provider error rate. Batches that fail are excluded.
