ai benchmarks
Real-world AI performance metrics
Gauntlet Composite updated
Gauntlet Science updated
Gauntlet Ship updated
Gauntlet Teach updated
Gauntlet Judgment updated
Artificial Analysis data updated
Each line is one model, climbing from its lowest to its highest reasoning effort: up scores higher, right costs more per task.
- Pick models in the Models list.
- Hover or tap a dot for its effort, score and cost.
- Cost runs on a log scale: each equal step right is the same multiple of price.
Every model ranked by the best score it reaches within your budget, with what that setting costs per task.
- Drag the budget to the most you’ll pay per task.
- Dots run low → max effort; rings are over budget.
- Tap the dot by a name to add it to Curves.
The cheapest way to reach each score. Each white step is a setting nothing cheaper matches; grey dots are beaten for less.
- Find your score on the left; read across to the line.
- The table shows what each next step up costs.
- Pick models to see what beats their settings.
How fast reaching a score is getting cheaper: each line is the lowest cost per task to reach it with what was out by each date.
- Lower is cheaper; each drop is a cheaper release.
- The right-hand label is today’s cheapest price.
- Intelligence, Coding or Agentic changes the score.
Every change Artificial Analysis made to the data, newest first.
- Click a model to see its past scores.
Gauntlet Science measures whether a model can do science on something it has never seen. Every run generates 24 worlds fresh, each with its own hidden law, so there is nothing to memorise. In each world the model reads noisy observations, runs 6 experiments it designs itself, states the law it found, then makes 20 predictions with intervals, half of them beyond anything it measured.
GPT-6.2 Astra sits 1.0× closer to 100 than Claude Fable 5.2, a paired gap of −0.5 points (95% −3.6 to +2.6).
- Ceiling scriptreferenceCeiling script, a reference script checked on every item: Gauntlet Science score 93
- 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Science score 90, the leader
- 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Science score 89, 1.0 times the distance to 100 of the model above, −0.5 points from the leader, 95% interval −3.6 to +2.6, not separated
- 3GPT-6.2 SolGPT-6.2 Sol: Gauntlet Science score 86, 1.3 times the distance to 100 of the model above, −3.8 points from the leader, 95% interval −5.6 to −1.9, separated
- 4GPT-6 AstraGPT-6 Astra: Gauntlet Science score 85, 1.0 times the distance to 100 of the model above, −4.3 points from the leader, 95% interval −7.0 to −1.7, separated
- 5Claude Opus 5.5Claude Opus 5.5: Gauntlet Science score 82, 1.2 times the distance to 100 of the model above, −7.1 points from the leader, 95% interval −11.5 to −2.7, separated
- 6Claude Fable 5.1Claude Fable 5.1: Gauntlet Science score 82, 1.0 times the distance to 100 of the model above, −7.4 points from the leader, 95% interval −12.3 to −2.5, separated
- 7Gemini 4 NeonGemini 4 Neon: Gauntlet Science score 81, 1.1 times the distance to 100 of the model above, −8.6 points from the leader, 95% interval −21.5 to +4.2, not separated
- 8GPT-6.1 SolGPT-6.1 Sol: Gauntlet Science score 81, 1.0 times the distance to 100 of the model above, −8.8 points from the leader, 95% interval −14.6 to −3.1, separated
- 9Gemini 4 CarbonGemini 4 Carbon: Gauntlet Science score 78, 1.1 times the distance to 100 of the model above, −11.5 points from the leader, 95% interval −24.4 to +1.5, not separated
- 10Muse WonderMuse Wonder: Gauntlet Science score 78, 1.0 times the distance to 100 of the model above, −11.7 points from the leader, 95% interval −17.7 to −5.6, separated
- 11Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Science score 73, 1.2 times the distance to 100 of the model above, −16.4 points from the leader, 95% interval −22.2 to −10.6, separated
- 12Gemini 4 ArgonGemini 4 Argon: Gauntlet Science score 66, 1.3 times the distance to 100 of the model above, −23.6 points from the leader, 95% interval −34.4 to −12.8, separated
- 13Grok 4.7Grok 4.7: Gauntlet Science score 64, 1.1 times the distance to 100 of the model above, −25.5 points from the leader, 95% interval −36.0 to −15.0, separated
- 14Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Science score 60, 1.1 times the distance to 100 of the model above, −29.6 points from the leader, 95% interval −38.5 to −20.7, separated
- 15GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Science score 59, 1.0 times the distance to 100 of the model above, −30.9 points from the leader, 95% interval −41.9 to −19.8, separated
- 16GPT-6 LunaGPT-6 Luna: Gauntlet Science score 44, 1.4 times the distance to 100 of the model above, −45.5 points from the leader, 95% interval −58.7 to −32.2, separated
- Floor scriptreferenceFloor script, a reference script checked on every item: Gauntlet Science score 37
Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.
- Ceiling scriptreferenceCeiling script, a reference script: score 93, 95% interval 90 to 9795% 90–97
- 1GPT-6.2 AstraGPT-6.2 Astra: score 90, 95% interval 83 to 96, the leader95% 83–96Leader
- 2Claude Fable 5.2Claude Fable 5.2: score 89, 95% interval 84 to 94, −0.5 points from the leader, 95% interval −3.6 to +2.6, not separated95% 84–94−0.595% −3.6 to +2.6Not separated
- 3GPT-6.2 SolGPT-6.2 Sol: score 86, 95% interval 79 to 92, −3.8 points from the leader, 95% interval −5.6 to −1.9, separated95% 79–92−3.895% −5.6 to −1.9● Separated
- 4GPT-6 AstraGPT-6 Astra: score 85, 95% interval 79 to 92, −4.3 points from the leader, 95% interval −7.0 to −1.7, separated95% 79–92−4.395% −7.0 to −1.7● Separated
- 5Claude Opus 5.5Claude Opus 5.5: score 82, 95% interval 74 to 91, −7.1 points from the leader, 95% interval −11.5 to −2.7, separated95% 74–91−7.195% −11.5 to −2.7● Separated
- 6Claude Fable 5.1Claude Fable 5.1: score 82, 95% interval 74 to 91, −7.4 points from the leader, 95% interval −12.3 to −2.5, separated95% 74–91−7.495% −12.3 to −2.5● Separated
- 7Gemini 4 NeonGemini 4 Neon: score 81, 95% interval 69 to 93, −8.6 points from the leader, 95% interval −21.5 to +4.2, not separated95% 69–93−8.695% −21.5 to +4.2Not separated
- 8GPT-6.1 SolGPT-6.1 Sol: score 81, 95% interval 73 to 88, −8.8 points from the leader, 95% interval −14.6 to −3.1, separated95% 73–88−8.895% −14.6 to −3.1● Separated
- 9Gemini 4 CarbonGemini 4 Carbon: score 78, 95% interval 65 to 91, −11.5 points from the leader, 95% interval −24.4 to +1.5, not separated95% 65–91−11.595% −24.4 to +1.5Not separated
- 10Muse WonderMuse Wonder: score 78, 95% interval 69 to 87, −11.7 points from the leader, 95% interval −17.7 to −5.6, separated95% 69–87−11.795% −17.7 to −5.6● Separated
- 11Claude Sonnet 5.5Claude Sonnet 5.5: score 73, 95% interval 65 to 81, −16.4 points from the leader, 95% interval −22.2 to −10.6, separated95% 65–81−16.495% −22.2 to −10.6● Separated
- 12Gemini 4 ArgonGemini 4 Argon: score 66, 95% interval 55 to 77, −23.6 points from the leader, 95% interval −34.4 to −12.8, separated95% 55–77−23.695% −34.4 to −12.8● Separated
- 13Grok 4.7Grok 4.7: score 64, 95% interval 51 to 77, −25.5 points from the leader, 95% interval −36.0 to −15.0, separated95% 51–77−25.595% −36.0 to −15.0● Separated
- 14Claude Haiku 5.5Claude Haiku 5.5: score 60, 95% interval 50 to 69, −29.6 points from the leader, 95% interval −38.5 to −20.7, separated95% 50–69−29.695% −38.5 to −20.7● Separated
- 15GPT-6.1 TerraGPT-6.1 Terra: score 59, 95% interval 48 to 69, −30.9 points from the leader, 95% interval −41.9 to −19.8, separated95% 48–69−30.995% −41.9 to −19.8● Separated
- 16GPT-6 LunaGPT-6 Luna: score 44, 95% interval 33 to 55, −45.5 points from the leader, 95% interval −58.7 to −32.2, separated95% 33–55−45.595% −58.7 to −32.2● Separated
- Floor scriptreferenceFloor script, a reference script: score 37, 95% interval 26 to 4795% 26–47
Code wrote each world and knows its law, so code scores every prediction against the hidden law: no judge reads an answer. A score of 100 is every prediction exact and certain; 0 is a miss or no answer. The worlds climb three rungs of difficulty (8, 8 and 8 worlds), and each score carries a 95% interval over the worlds. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are scripts, not models. The floor script runs random experiments and fits a straight line, so a model below it did no better than that. The ceiling script spreads its experiments to the edges and picks the best-fitting law from the families the worlds are drawn from, a practical top.
Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (separation, headroom, stability, length, cap, no cheap twin and effort). This run fails 1 of them (stability), so read its ranking as provisional.
Not ranked, because a model is ranked only when it finished every world: Kimi (2 of 24 cells never reached an answer: the provider failed).
Worlds that ended at a time or output limit are scored as unanswered, which lowers a row: Grok 4.7 on 3 of 24, Gemini 4 Neon on 2 of 24, Gemini 4 Carbon on 2 of 24 and Muse Wonder on 1 of 24.
Graded 2026-11-20; run f8b224a2, seed 20261112, protocol gauntlet-discovery-v3. Full methodology
Gauntlet Ship measures whether a model can finish real work. Each of the 55 items is a real task from Unsupervised Learning’s own working sessions, across 16 parts of a codebase the model has never seen. The model gets the working conversation as its ask and the repository as it stood before the fix, then has 30 steps of one plain tool loop to read, change and test the code before it says whether it is done.
Claude Fable 5.2 makes 1.1× fewer misses than Claude Fable 5.1, a paired gap of −3.6 points (95% −15.0 to +7.7).
- Accepted fixreferenceAccepted fix, a reference script checked on every item: Gauntlet Ship score 100
- 1Claude Fable 5.2Claude Fable 5.2: Gauntlet Ship score 71, hidden checks 83%, false done 19%, the leader
- 2Claude Fable 5.1Claude Fable 5.1: Gauntlet Ship score 67, hidden checks 77%, false done 30%, 1.1 times the misses of the model above, −3.6 points from the leader, 95% interval −15.0 to +7.7, not separated
- 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Ship score 66, hidden checks 74%, false done 26%, 1.1 times the misses of the model above, −5.5 points from the leader, 95% interval −17.3 to +6.4, not separated
- 4GPT-6.1 SolGPT-6.1 Sol: Gauntlet Ship score 64, hidden checks 76%, false done 30%, 1.1 times the misses of the model above, −7.3 points from the leader, 95% interval −19.6 to +5.0, not separated
- 5GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Ship score 64, hidden checks 76%, false done 31%, 1.0 times the misses of the model above, −7.3 points from the leader, 95% interval −18.5 to +3.9, not separated
- 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Ship score 60, hidden checks 72%, false done 39%, 1.1 times the misses of the model above, −10.9 points from the leader, 95% interval −21.9 to +0.1, not separated
- 7GPT-6.2 SolGPT-6.2 Sol: Gauntlet Ship score 55, hidden checks 68%, false done 40%, 1.1 times the misses of the model above, −16.4 points from the leader, 95% interval −26.2 to −6.5, separated
- 8Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Ship score 46, hidden checks 61%, false done 46%, 1.2 times the misses of the model above, −25.5 points from the leader, 95% interval −38.1 to −12.8, separated
- 9GPT-6 AstraGPT-6 Astra: Gauntlet Ship score 40, hidden checks 55%, false done 35%, 1.1 times the misses of the model above, −30.9 points from the leader, 95% interval −44.2 to −17.6, separated
- 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Ship score 40, hidden checks 52%, false done 40%, 1.0 times the misses of the model above, −30.9 points from the leader, 95% interval −45.2 to −16.6, separated
- No changereferenceNo change, a reference script checked on every item: Gauntlet Ship score 0
Placed by gap: equal distances are equal ratios of misses, across only the models shown.
- Accepted fixreferenceAccepted fix, a reference script: Gauntlet Ship score 100, 95% interval 100 to 10095% 100–100
- 1Claude Fable 5.2Hidden checks 83%False done 19%Claude Fable 5.2: Gauntlet Ship score 71, 95% interval 59 to 83, hidden checks 83%, false done 19%, the leader95% 59–83Leader
- 2Claude Fable 5.1Hidden checks 77%False done 30%Claude Fable 5.1: Gauntlet Ship score 67, 95% interval 55 to 80, hidden checks 77%, false done 30%, −3.6 points from the leader, 95% interval −15.0 to +7.7, not separated95% 55–80−3.695% −15.0 to +7.7Not separated
- 3Claude Opus 5.5Hidden checks 74%False done 26%Claude Opus 5.5: Gauntlet Ship score 66, 95% interval 53 to 78, hidden checks 74%, false done 26%, −5.5 points from the leader, 95% interval −17.3 to +6.4, not separated95% 53–78−5.595% −17.3 to +6.4Not separated
- 4GPT-6.1 SolHidden checks 76%False done 30%GPT-6.1 Sol: Gauntlet Ship score 64, 95% interval 51 to 77, hidden checks 76%, false done 30%, −7.3 points from the leader, 95% interval −19.6 to +5.0, not separated95% 51–77−7.395% −19.6 to +5.0Not separated
- 5GPT-6.2 AstraHidden checks 76%False done 31%GPT-6.2 Astra: Gauntlet Ship score 64, 95% interval 51 to 77, hidden checks 76%, false done 31%, −7.3 points from the leader, 95% interval −18.5 to +3.9, not separated95% 51–77−7.395% −18.5 to +3.9Not separated
- 6Claude Sonnet 5.5Hidden checks 72%False done 39%Claude Sonnet 5.5: Gauntlet Ship score 60, 95% interval 47 to 73, hidden checks 72%, false done 39%, −10.9 points from the leader, 95% interval −21.9 to +0.1, not separated95% 47–73−10.995% −21.9 to +0.1Not separated
- 7GPT-6.2 SolHidden checks 68%False done 40%GPT-6.2 Sol: Gauntlet Ship score 55, 95% interval 41 to 68, hidden checks 68%, false done 40%, −16.4 points from the leader, 95% interval −26.2 to −6.5, separated95% 41–68−16.495% −26.2 to −6.5● Separated
- 8Claude Haiku 5.5Hidden checks 61%False done 46%Claude Haiku 5.5: Gauntlet Ship score 46, 95% interval 32 to 59, hidden checks 61%, false done 46%, −25.5 points from the leader, 95% interval −38.1 to −12.8, separated95% 32–59−25.595% −38.1 to −12.8● Separated
- 9GPT-6 AstraHidden checks 55%False done 35%GPT-6 Astra: Gauntlet Ship score 40, 95% interval 27 to 53, hidden checks 55%, false done 35%, −30.9 points from the leader, 95% interval −44.2 to −17.6, separated95% 27–53−30.995% −44.2 to −17.6● Separated
- 10GPT-6.1 TerraHidden checks 52%False done 40%GPT-6.1 Terra: Gauntlet Ship score 40, 95% interval 27 to 53, hidden checks 52%, false done 40%, −30.9 points from the leader, 95% interval −45.2 to −16.6, separated95% 27–53−30.995% −45.2 to −16.6● Separated
- No changereferenceNo change, a reference script: Gauntlet Ship score 0, 95% interval 0 to 095% 0–0
Hidden checks, written from that conversation, decide each item. A check is kept only when it fails on the code before the accepted fix and passes after it, and checks are screened so they test nothing the conversation left open. An item is shipped when every hidden check passes and no test that passed before breaks. Code runs the checks and makes every number: no judge reads the work. The Gauntlet Ship score is the share of items shipped, with a 95% interval over the items. Hidden checks is the share of hidden checks a model passed, and false done is the share of its “done” claims that did not ship. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are scripts, not models. No change finishes at once and ships nothing, so a model at that row changed nothing that mattered. The accepted fix writes the fix that was actually accepted, and every item it does not ship is dropped, so each item counted here can be shipped.
Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (items valid, separation, headroom, stability, no cheap twin, length and effort). This run fails 1 of them (stability), so read its ranking as provisional.
Argon ran but is left out of this ranking: 47 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked. Carbon ran but is left out of this ranking: 54 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked. Neon ran but is left out of this ranking: 50 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked.
Grok 4.7 played but is not ranked (5 of 55 cells never reached an answer: the provider failed). Muse Wonder played but is not ranked (54 of 55 cells never reached an answer: the provider refused for rate or quota).
The items are private work, so this page shows counts and scores only: never an item’s conversation, its code, its checks or a model’s transcript.
Graded 2026-11-20; run f401c811, item set v4, protocol gauntlet-ship-v1. Full methodology
Gauntlet Composite puts Unsupervised Learning’s own benchmarks together. Each dot is one model’s plain mean of its three scores on 0–100: Gauntlet Science, Gauntlet Ship and Gauntlet Teach. The small marks on each row are those parts. Only the 10 models every one of these benchmarks ran are ranked, so every dot is made of the same parts. Every model runs at high effort; a model whose API has no effort setting runs at its own default.
GPT-6.2 Astra sits 1.1× closer to 100 than Claude Fable 5.2.
- 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Composite 81, the mean of Gauntlet Science 90, Gauntlet Ship 64 and Gauntlet Teach 91, the leader
- 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Composite 80, the mean of Gauntlet Science 89, Gauntlet Ship 71 and Gauntlet Teach 81, 1.1 times the distance to 100 of the model above
- 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Composite 76, the mean of Gauntlet Science 82, Gauntlet Ship 66 and Gauntlet Teach 81, 1.2 times the distance to 100 of the model above
- 4Claude Fable 5.1Claude Fable 5.1: Gauntlet Composite 74, the mean of Gauntlet Science 82, Gauntlet Ship 67 and Gauntlet Teach 73, 1.1 times the distance to 100 of the model above
- 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Composite 73, the mean of Gauntlet Science 86, Gauntlet Ship 55 and Gauntlet Teach 80, 1.0 times the distance to 100 of the model above
- 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Composite 72, the mean of Gauntlet Science 73, Gauntlet Ship 60 and Gauntlet Teach 82, 1.1 times the distance to 100 of the model above
- 7GPT-6.1 SolGPT-6.1 Sol: Gauntlet Composite 71, the mean of Gauntlet Science 81, Gauntlet Ship 64 and Gauntlet Teach 69, 1.0 times the distance to 100 of the model above
- 8GPT-6 AstraGPT-6 Astra: Gauntlet Composite 67, the mean of Gauntlet Science 85, Gauntlet Ship 40 and Gauntlet Teach 76, 1.1 times the distance to 100 of the model above
- 9Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Composite 56, the mean of Gauntlet Science 60, Gauntlet Ship 46 and Gauntlet Teach 62, 1.3 times the distance to 100 of the model above
- 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Composite 44, the mean of Gauntlet Science 59, Gauntlet Ship 40 and Gauntlet Teach 34, 1.3 times the distance to 100 of the model above
Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.
Gauntlet CompositeGauntlet ScienceGauntlet ShipGauntlet Teach
- 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Composite 81, the mean of Gauntlet Science 90, Gauntlet Ship 64 and Gauntlet Teach 91
- 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Composite 80, the mean of Gauntlet Science 89, Gauntlet Ship 71 and Gauntlet Teach 81
- 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Composite 76, the mean of Gauntlet Science 82, Gauntlet Ship 66 and Gauntlet Teach 81
- 4Claude Fable 5.1Claude Fable 5.1: Gauntlet Composite 74, the mean of Gauntlet Science 82, Gauntlet Ship 67 and Gauntlet Teach 73
- 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Composite 73, the mean of Gauntlet Science 86, Gauntlet Ship 55 and Gauntlet Teach 80
- 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Composite 72, the mean of Gauntlet Science 73, Gauntlet Ship 60 and Gauntlet Teach 82
- 7GPT-6.1 SolGPT-6.1 Sol: Gauntlet Composite 71, the mean of Gauntlet Science 81, Gauntlet Ship 64 and Gauntlet Teach 69
- 8GPT-6 AstraGPT-6 Astra: Gauntlet Composite 67, the mean of Gauntlet Science 85, Gauntlet Ship 40 and Gauntlet Teach 76
- 9Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Composite 56, the mean of Gauntlet Science 60, Gauntlet Ship 46 and Gauntlet Teach 62
- 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Composite 44, the mean of Gauntlet Science 59, Gauntlet Ship 40 and Gauntlet Teach 34
Not ranked, because not every benchmark ran them: Gemini 4 Argon (no Gauntlet Ship score), Gemini 4 Carbon (no Gauntlet Ship score), Gemini 4 Neon (no Gauntlet Ship score), GPT-6 Luna (no Gauntlet Ship score), Grok 4.7 (no Gauntlet Ship and Gauntlet Teach score) and Muse Wonder (no Gauntlet Ship and Gauntlet Teach score).
Each part comes from its own benchmark’s published run; nothing here is measured anew. The current runs of Gauntlet Science and Gauntlet Ship each fail one or more of their pre-registered checks, so those parts, and this ranking with them, are provisional.
Made from the published Gauntlet Science, Gauntlet Ship and Gauntlet Teach runs, newest graded 2026-11-20. Methodology
Gauntlet Teach measures whether a model can explain what it found so that someone else can use it. Each model gets the evidence for 12 invented Gauntlet Science worlds: the field records and the same designed experiments for every model. For each world it writes a plain-words explanation of the law it finds, at most 100 words, with no formula or math notation.
GPT-6.2 Astra sits 1.9× closer to 100 than Claude Sonnet 5.5, a paired gap of −8.2 points (95% −18.7 to +2.3).
- True lawreferenceTrue law, a reference script checked on every item: Gauntlet Teach score 98
- 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Teach score 91, its own Gauntlet Science score 86, check student 89, the leader, by a clear lead
- 2Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Teach score 82, its own Gauntlet Science score 71, check student 82, 1.9 times the distance to 100 of the model above, −8.2 points from the leader, 95% interval −18.7 to +2.3, not separated
- 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Teach score 81, its own Gauntlet Science score 78, check student 80, 1.1 times the distance to 100 of the model above, −9.5 points from the leader, 95% interval −26.3 to +7.4, not separated
- 4Claude Fable 5.2Claude Fable 5.2: Gauntlet Teach score 81, its own Gauntlet Science score 88, check student 80, 1.0 times the distance to 100 of the model above, −9.9 points from the leader, 95% interval −22.7 to +2.9, not separated
- 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Teach score 80, its own Gauntlet Science score 84, check student 79, 1.0 times the distance to 100 of the model above, −10.9 points from the leader, 95% interval −20.6 to −1.1, separated
- 6GPT-6 AstraGPT-6 Astra: Gauntlet Teach score 76, its own Gauntlet Science score 82, check student 71, 1.2 times the distance to 100 of the model above, −14.4 points from the leader, 95% interval −31.0 to +2.3, not separated
- 7Claude Fable 5.1Claude Fable 5.1: Gauntlet Teach score 73, its own Gauntlet Science score 75, check student 71, 1.1 times the distance to 100 of the model above, −17.3 points from the leader, 95% interval −32.2 to −2.4, separated
- 8Gemini 4 NeonGemini 4 Neon: Gauntlet Teach score 72, its own Gauntlet Science score 75, check student 72, 1.0 times the distance to 100 of the model above, −18.4 points from the leader, 95% interval −36.8 to +0.0, not separated
- 9GPT-6.1 SolGPT-6.1 Sol: Gauntlet Teach score 69, its own Gauntlet Science score 79, check student 66, 1.1 times the distance to 100 of the model above, −21.9 points from the leader, 95% interval −41.2 to −2.7, separated
- 10Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Teach score 62, its own Gauntlet Science score 64, check student 65, 1.2 times the distance to 100 of the model above, −28.9 points from the leader, 95% interval −48.4 to −9.3, separated
- 11Gemini 4 ArgonGemini 4 Argon: Gauntlet Teach score 58, its own Gauntlet Science score 68, check student 56, 1.1 times the distance to 100 of the model above, −33.0 points from the leader, 95% interval −54.6 to −11.4, separated
- 12Gemini 4 CarbonGemini 4 Carbon: Gauntlet Teach score 57, its own Gauntlet Science score 75, check student 57, 1.0 times the distance to 100 of the model above, −33.1 points from the leader, 95% interval −57.9 to −8.4, separated
- 13GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Teach score 34, its own Gauntlet Science score 61, check student 38, 1.5 times the distance to 100 of the model above, −56.2 points from the leader, 95% interval −81.5 to −31.0, separated
- 14GPT-6 LunaGPT-6 Luna: Gauntlet Teach score 17, its own Gauntlet Science score 45, check student 15, 1.3 times the distance to 100 of the model above, −73.1 points from the leader, 95% interval −91.7 to −54.6, separated
- No explanationreferenceNo explanation, a reference script checked on every item: Gauntlet Teach score 6
Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.
Gauntlet Teach score, with its 95% intervalThe model’s own Gauntlet Science score on the same worlds
- True lawreferenceTrue law, a reference row: Gauntlet Teach score 98, 95% interval 98 to 9995% 98–99
- 1GPT-6.2 Astra◇ Gauntlet Science 86Check student 89GPT-6.2 Astra: Gauntlet Teach score 91, 95% interval 85 to 96, its own Gauntlet Science score 86, check student 89, the leader95% 85–96Leader
- 2Claude Sonnet 5.5◇ Gauntlet Science 71Check student 82Claude Sonnet 5.5: Gauntlet Teach score 82, 95% interval 72 to 92, its own Gauntlet Science score 71, check student 82, −8.2 points from the leader, 95% interval −18.7 to +2.3, not separated95% 72–92−8.295% −18.7 to +2.3Not separated
- 3Claude Opus 5.5◇ Gauntlet Science 78Check student 80Claude Opus 5.5: Gauntlet Teach score 81, 95% interval 66 to 96, its own Gauntlet Science score 78, check student 80, −9.5 points from the leader, 95% interval −26.3 to +7.4, not separated95% 66–96−9.595% −26.3 to +7.4Not separated
- 4Claude Fable 5.2◇ Gauntlet Science 88Check student 80Claude Fable 5.2: Gauntlet Teach score 81, 95% interval 68 to 93, its own Gauntlet Science score 88, check student 80, −9.9 points from the leader, 95% interval −22.7 to +2.9, not separated95% 68–93−9.995% −22.7 to +2.9Not separated
- 5GPT-6.2 Sol◇ Gauntlet Science 84Check student 79GPT-6.2 Sol: Gauntlet Teach score 80, 95% interval 69 to 90, its own Gauntlet Science score 84, check student 79, −10.9 points from the leader, 95% interval −20.6 to −1.1, separated95% 69–90−10.995% −20.6 to −1.1● Separated
- 6GPT-6 Astra◇ Gauntlet Science 82Check student 71GPT-6 Astra: Gauntlet Teach score 76, 95% interval 61 to 92, its own Gauntlet Science score 82, check student 71, −14.4 points from the leader, 95% interval −31.0 to +2.3, not separated95% 61–92−14.495% −31.0 to +2.3Not separated
- 7Claude Fable 5.1◇ Gauntlet Science 75Check student 71Claude Fable 5.1: Gauntlet Teach score 73, 95% interval 59 to 88, its own Gauntlet Science score 75, check student 71, −17.3 points from the leader, 95% interval −32.2 to −2.4, separated95% 59–88−17.395% −32.2 to −2.4● Separated
- 8Gemini 4 Neon◇ Gauntlet Science 75Check student 72Gemini 4 Neon: Gauntlet Teach score 72, 95% interval 54 to 90, its own Gauntlet Science score 75, check student 72, −18.4 points from the leader, 95% interval −36.8 to 0.0, not separated95% 54–90−18.495% −36.8 to 0.0Not separated
- 9GPT-6.1 Sol◇ Gauntlet Science 79Check student 66GPT-6.1 Sol: Gauntlet Teach score 69, 95% interval 49 to 88, its own Gauntlet Science score 79, check student 66, −21.9 points from the leader, 95% interval −41.2 to −2.7, separated95% 49–88−21.995% −41.2 to −2.7● Separated
- 10Claude Haiku 5.5◇ Gauntlet Science 64Check student 65Claude Haiku 5.5: Gauntlet Teach score 62, 95% interval 44 to 80, its own Gauntlet Science score 64, check student 65, −28.9 points from the leader, 95% interval −48.4 to −9.3, separated95% 44–80−28.995% −48.4 to −9.3● Separated
- 11Gemini 4 Argon◇ Gauntlet Science 68Check student 56Gemini 4 Argon: Gauntlet Teach score 58, 95% interval 37 to 78, its own Gauntlet Science score 68, check student 56, −33.0 points from the leader, 95% interval −54.6 to −11.4, separated95% 37–78−33.095% −54.6 to −11.4● Separated
- 12Gemini 4 Carbon◇ Gauntlet Science 75Check student 57Gemini 4 Carbon: Gauntlet Teach score 57, 95% interval 34 to 81, its own Gauntlet Science score 75, check student 57, −33.1 points from the leader, 95% interval −57.9 to −8.4, separated95% 34–81−33.195% −57.9 to −8.4● Separated
- 13GPT-6.1 Terra◇ Gauntlet Science 61Check student 38GPT-6.1 Terra: Gauntlet Teach score 34, 95% interval 12 to 57, its own Gauntlet Science score 61, check student 38, −56.2 points from the leader, 95% interval −81.5 to −31.0, separated95% 12–57−56.295% −81.5 to −31.0● Separated
- 14GPT-6 Luna◇ Gauntlet Science 45Check student 15GPT-6 Luna: Gauntlet Teach score 17, 95% interval 2 to 32, its own Gauntlet Science score 45, check student 15, −73.1 points from the leader, 95% interval −91.7 to −54.6, separated95% 2–32−73.195% −91.7 to −54.6● Separated
- No explanationreferenceNo explanation, a reference row: Gauntlet Teach score 6, 95% interval 1 to 1095% 1–10
A fixed student, Claude Sonnet 5.5 at low effort, never sees the data. It reads each explanation and turns it into a formula; code computes that formula at the world’s held-out settings and scores the predictions with Gauntlet Science’s own scoring, from 0 to 100. No judge reads an explanation, and the student’s own knowledge cannot help, because the worlds were invented. Gemini 4 Argon reads the same explanations as a check student, and the two must rank the models alike. A world a model did not finish explaining in time is scored the way an empty explanation is, as the floor student; only a provider failure leaves a model unranked. Each score carries a 95% interval over the worlds, and a model’s gap to the leader is paired world by world. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are the same student without a model’s explanation. With no explanation it can only guess, so a model at that row taught it nothing. Handed the true law, it shows how far a perfect explanation could take it. The open diamond on each row is that model’s own Gauntlet Science score on the same 12 worlds, from Gauntlet Science run e05093a6, so a model that finds a law well but explains it poorly shows as a gap between the two.
Before any model ran, we fixed 5 checks the run must pass to count as a valid measurement (instrument, separation, student agreement, length and references reached an answer). This run passes all 5. One more figure is reported, not gated: not a copy of Gauntlet Science (lane-order Spearman with own Gauntlet Science score 0.69).
Not ranked, each for its own reason: Kimi (12 of 12 cells never reached an answer: the provider refused for rate or quota), Muse Wonder (5 of 12 cells never reached an answer: the provider refused for rate or quota on 1, and failed on 4 more) and Grok 4.7 (not run in this Gauntlet Teach run).
Worlds that ended at a time or output limit are scored as unanswered, which lowers a row: Gemini 4 Carbon on 3 of 12, Gemini 4 Neon on 2 of 12, Gemini 4 Argon on 2 of 12 and Claude Fable 5.2 on 1 of 12.
The worlds, the explanations and the students’ formulas stay private, so this page shows scores only.
Graded 2026-11-20; run 5775e008 on Gauntlet Science run e05093a6, 12 worlds, at most 100 words, protocol gauntlet-teach-v2. Full methodology
Gauntlet Judgment measures whether a model can find and fix a bug from the evidence alone. Each of the 63 items is one of Unsupervised Learning’s own tested source modules with two bugs planted in it. The model sees only the cleaned output of the failing tests and the folder’s file list with each file’s exported names, may open one file, and returns exact edits. Each model call has up to 15 minutes. Code applies them in a sandbox, and the hidden tests decide.
Gemini 4 Neon makes 1.2× fewer misses than GPT-6.1 Sol, a paired gap of −3.2 points (95% −12.0 to +5.7).
- Bugs revertedreferenceBugs reverted, a reference script checked on every item: Gauntlet Judgment score 100
- 1Gemini 4 NeonGemini 4 Neon: Gauntlet Judgment score 84, right file 92%, the leader
- 2GPT-6.1 SolGPT-6.1 Sol: Gauntlet Judgment score 81, right file 95%, 1.2 times the misses of the model above, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated
- 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Judgment score 81, right file 94%, 1.0 times the misses of the model above, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated
- 4Claude Fable 5.2Claude Fable 5.2: Gauntlet Judgment score 81, right file 95%, 1.0 times the misses of the model above, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated
- 5Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Judgment score 75, right file 94%, 1.3 times the misses of the model above, −9.5 points from the leader, 95% interval −18.1 to −1.0, separated
- 6Claude Fable 5.1Claude Fable 5.1: Gauntlet Judgment score 73, right file 89%, 1.1 times the misses of the model above, −11.1 points from the leader, 95% interval −21.1 to −1.1, separated
- 7Gemini 4 ArgonGemini 4 Argon: Gauntlet Judgment score 71, right file 95%, 1.1 times the misses of the model above, −12.7 points from the leader, 95% interval −22.1 to −3.3, separated
- 8Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Judgment score 70, right file 97%, 1.1 times the misses of the model above, −14.3 points from the leader, 95% interval −25.0 to −3.6, separated
- No editreferenceNo edit, a reference script checked on every item: Gauntlet Judgment score 0
Placed by gap: equal distances are equal ratios of misses, across only the models shown.
- Bugs revertedreferenceBugs reverted, a reference script checked on every item: Gauntlet Judgment score 100Every item
- 1Gemini 4 NeonRight file 92%Gemini 4 Neon: Gauntlet Judgment score 84, 95% interval 75 to 93, right file 92%, the leader95% 75–93Leader
- 2GPT-6.1 SolRight file 95%GPT-6.1 Sol: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 95%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
- 3Claude Opus 5.5Right file 94%Claude Opus 5.5: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 94%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
- 4Claude Fable 5.2Right file 95%Claude Fable 5.2: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 95%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
- 5Claude Sonnet 5.5Right file 94%Claude Sonnet 5.5: Gauntlet Judgment score 75, 95% interval 64 to 85, right file 94%, −9.5 points from the leader, 95% interval −18.1 to −1.0, separated95% 64–85−9.595% −18.1 to −1.0● Separated
- 6Claude Fable 5.1Right file 89%Claude Fable 5.1: Gauntlet Judgment score 73, 95% interval 62 to 84, right file 89%, −11.1 points from the leader, 95% interval −21.1 to −1.1, separated95% 62–84−11.195% −21.1 to −1.1● Separated
- 7Gemini 4 ArgonRight file 95%Gemini 4 Argon: Gauntlet Judgment score 71, 95% interval 60 to 83, right file 95%, −12.7 points from the leader, 95% interval −22.1 to −3.3, separated95% 60–83−12.795% −22.1 to −3.3● Separated
- 8Claude Haiku 5.5Right file 97%Claude Haiku 5.5: Gauntlet Judgment score 70, 95% interval 58 to 81, right file 97%, −14.3 points from the leader, 95% interval −25.0 to −3.6, separated95% 58–81−14.395% −25.0 to −3.6● Separated
- No editreferenceNo edit, a reference script checked on every item: Gauntlet Judgment score 0Every item
An item is fixed when every hidden test passes after the edits. Code makes every number: no judge reads the work. The Gauntlet Judgment score is the share of items fixed, with a 95% interval over the items; the gap to the leader is paired item by item. Right file is the share of items where the file the model opened was the one holding the bugs. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are scripts, not models, checked on every item: no edit fixes nothing, and reverting the planted bugs fixes everything, so every item counted can be fixed.
Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (instrument, no cheap twin, separation, headroom, stability, length and reproducibility). This run fails 2 (stability and reproducibility), so read its ranking as provisional.
Two companion floor checks run too: Brake, refusing a destructive command when the deciding fact is buried in tool output, and Receipt, reading a requirement out of a long log. Every model scores 100 on both so far, so they do not separate models and are not ranked.
The items are private code, so this page shows counts and scores only: never an item’s module, its code, the planted bugs, the failing output or a model’s reply.
Graded 2026-11-20; run e73a52c0, item set v11, protocol gauntlet-judgment-patch-v2. Not part of Gauntlet Composite. Full methodology
LMArena measures which answers people prefer. Anyone can put one question to two anonymous models and vote for the better reply, and millions of those votes become one rating per model. Artificial Analysis and the Gauntlet score answers by code against known answers; neither asks people which answer they would rather read.
LMArena · LMArena text score · data of · CC BY 4.0
Gemini 4 Carbon · high leads Gemini 4 Argon · high by 9 rating points.
- 1Gemini 4 Carbon · highGemini 4 Carbon · high: LMArena text score 1538, the leader
- 2Gemini 4 Argon · highGemini 4 Argon · high: LMArena text score 1529, 9 rating points below the model above
- 3Claude Opus 5.5 · highClaude Opus 5.5 · high: LMArena text score 1514, 16 rating points below the model above
- 4Claude Fable 5.1 · maxClaude Fable 5.1 · max: LMArena text score 1512, 2 rating points below the model above
- 5Claude Opus 5 · maxClaude Opus 5 · max: LMArena text score 1507, 6 rating points below the model above
- 6Muse Wonder · maxMuse Wonder · max: LMArena text score 1503, 4 rating points below the model above
- 7Claude Opus 5 · highClaude Opus 5 · high: LMArena text score 1502, 1 rating points below the model above
- 8Gemini 4 Hydrogen · highGemini 4 Hydrogen · high: LMArena text score 1502, 0 rating points below the model above
- 9Gemini 3.8 Flash · highGemini 3.8 Flash · high: LMArena text score 1500, 3 rating points below the model above
- 10Muse Spark 1.3 · maxMuse Spark 1.3 · max: LMArena text score 1492, 8 rating points below the model above
- 11MiMo-V2.6-ProMiMo-V2.6-Pro: LMArena text score 1492, 0 rating points below the model above
- 12GLM-5.3 · maxGLM-5.3 · max: LMArena text score 1475, 17 rating points below the model above
- 13Kimi K3 · maxKimi K3 · max: LMArena text score 1475, 0 rating points below the model above
- 14GLM-5.3-FlashGLM-5.3-Flash: LMArena text score 1472, 3 rating points below the model above
- 15Claude Sonnet 5.5 · xhighClaude Sonnet 5.5 · xhigh: LMArena text score 1469, 3 rating points below the model above
- 16Kimi K3.1 Pro · maxKimi K3.1 Pro · max: LMArena text score 1459, 10 rating points below the model above
- 17MiMo-V2.6-FlashMiMo-V2.6-Flash: LMArena text score 1451, 8 rating points below the model above
- 18GPT-5.6 Terra · xhighGPT-5.6 Terra · xhigh: LMArena text score 1449, 2 rating points below the model above
- 19GPT-6.1 Sol · maxGPT-6.1 Sol · max: LMArena text score 1445, 4 rating points below the model above
- 20Step 5 PreviewStep 5 Preview: LMArena text score 1443, 1 rating points below the model above
- 21InklingInkling: LMArena text score 1443, 0 rating points below the model above
- 22Gemma 4 31BGemma 4 31B: LMArena text score 1443, 1 rating points below the model above
- 23GPT-6 Astra · maxGPT-6 Astra · max: LMArena text score 1440, 3 rating points below the model above
- 24Qwen3.8 27BQwen3.8 27B: LMArena text score 1437, 3 rating points below the model above
- 25Gemini 3.5 Flash-LiteGemini 3.5 Flash-Lite: LMArena text score 1435, 2 rating points below the model above
- 26GPT-5.6 Luna · xhighGPT-5.6 Luna · xhigh: LMArena text score 1431, 4 rating points below the model above
- 27Grok 4.6 · highGrok 4.6 · high: LMArena text score 1430, 1 rating points below the model above
- 28MiniMax-M3MiniMax-M3: LMArena text score 1428, 2 rating points below the model above
- 29Mistral Medium 3.5Mistral Medium 3.5: LMArena text score 1421, 7 rating points below the model above
- 30Inkling SmallInkling Small: LMArena text score 1414, 7 rating points below the model above
- 31Mistral Large 4Mistral Large 4: LMArena text score 1411, 3 rating points below the model above
- 32Inkling 2 SmallInkling 2 Small: LMArena text score 1403, 9 rating points below the model above
- 33Grok 4.7 · xhighGrok 4.7 · xhigh: LMArena text score 1400, 3 rating points below the model above
- 34Claude Haiku 5.5 · highClaude Haiku 5.5 · high: LMArena text score 1398, 2 rating points below the model above
- 35GPT-6 Sol · maxGPT-6 Sol · max: LMArena text score 1397, 1 rating points below the model above
- 36Muse GlimmerMuse Glimmer: LMArena text score 1389, 8 rating points below the model above
- 37GPT-6 Luna · maxGPT-6 Luna · max: LMArena text score 1388, 0 rating points below the model above
- 38Solar Pro 4Solar Pro 4: LMArena text score 1388, 0 rating points below the model above
- 39gpt-oss-120bgpt-oss-120b: LMArena text score 1366, 23 rating points below the model above
- 40Command A+ · 03-2025Command A+ · 03-2025: LMArena text score 1331, 35 rating points below the model above
- 41Granite 4.2 8BGranite 4.2 8B: LMArena text score 1319, 12 rating points below the model above
- 42Granite 4.2 3BGranite 4.2 3B: LMArena text score 1292, 26 rating points below the model above
Placed by gap: in rating points, across only the models shown.
- 1Gemini 4 Carbon · high#1 of 421 on LMArenaGemini 4 Carbon · high: LMArena 1538, interval 1529–1548, 1 of 421 on LMArena, listed by LMArena as Gemini 4 Carbon (High)Interval 1529–1548
- 2Gemini 4 Argon · high#2 of 421 on LMArenaGemini 4 Argon · high: LMArena 1529, interval 1524–1535, 2 of 421 on LMArena, listed by LMArena as Gemini 4 Argon (High)Interval 1524–1535
- 3Claude Opus 5.5 · high#3 of 421 on LMArenaClaude Opus 5.5 · high: LMArena 1514, interval 1508–1520, 3 of 421 on LMArena, listed by LMArena as Claude Opus 5.5 (High)Interval 1508–1520
- 4Claude Fable 5.1 · max#4 of 421 on LMArenaClaude Fable 5.1 · max: LMArena 1512, interval 1507–1518, 4 of 421 on LMArena, listed by LMArena as Claude Fable 5.1 (Max)Interval 1507–1518
- 5Claude Opus 5 · max#5 of 421 on LMArenaClaude Opus 5 · max: LMArena 1507, interval 1502–1511, 5 of 421 on LMArena, listed by LMArena as Claude Opus 5 (Max)Interval 1502–1511
- 6Muse Wonder · max#6 of 421 on LMArenaMuse Wonder · max: LMArena 1503, interval 1497–1509, 6 of 421 on LMArena, listed by LMArena as Muse Wonder (Max)Interval 1497–1509
- 7Claude Opus 5 · high#7 of 421 on LMArenaClaude Opus 5 · high: LMArena 1502, interval 1499–1506, 7 of 421 on LMArena, listed by LMArena as Claude Opus 5 (High)Interval 1499–1506
- 8Gemini 4 Hydrogen · high#8 of 421 on LMArenaGemini 4 Hydrogen · high: LMArena 1502, interval 1493–1511, 8 of 421 on LMArena, listed by LMArena as Gemini 4 Hydrogen (High)Interval 1493–1511
- 9Gemini 3.8 Flash · high#10 of 421 on LMArenaGemini 3.8 Flash · high: LMArena 1500, interval 1495–1504, 10 of 421 on LMArena, listed by LMArena as Gemini 3.8 Flash (High)Interval 1495–1504
- 10Muse Spark 1.3 · max#13 of 421 on LMArenaMuse Spark 1.3 · max: LMArena 1492, interval 1487–1497, 13 of 421 on LMArena, listed by LMArena as Muse Spark 1.3 (Max)Interval 1487–1497
- 11MiMo-V2.6-Pro#14 of 421 on LMArenaMiMo-V2.6-Pro: LMArena 1492, interval 1482–1501, 14 of 421 on LMArena, listed by LMArena as MiMo-V2.6-ProInterval 1482–1501
- 12GLM-5.3 · max#26 of 421 on LMArenaGLM-5.3 · max: LMArena 1475, interval 1470–1480, 26 of 421 on LMArena, listed by LMArena as GLM-5.3 (Max)Interval 1470–1480
- 13Kimi K3 · max#28 of 421 on LMArenaKimi K3 · max: LMArena 1475, interval 1470–1479, 28 of 421 on LMArena, listed by LMArena as Kimi K3 (Max)Interval 1470–1479
- 14GLM-5.3-Flash#30 of 421 on LMArenaGLM-5.3-Flash: LMArena 1472, interval 1467–1476, 30 of 421 on LMArena, listed by LMArena as GLM-5.3-FlashInterval 1467–1476
- 15Claude Sonnet 5.5 · xhigh#34 of 421 on LMArenaClaude Sonnet 5.5 · xhigh: LMArena 1469, interval 1463–1475, 34 of 421 on LMArena, listed by LMArena as Claude Sonnet 5.5 (xHigh)Interval 1463–1475
- 16Kimi K3.1 Pro · max#43 of 421 on LMArenaKimi K3.1 Pro · max: LMArena 1459, interval 1449–1469, 43 of 421 on LMArena, listed by LMArena as Kimi K3.1 Pro (Max)Interval 1449–1469
- 17MiMo-V2.6-Flash#55 of 421 on LMArenaMiMo-V2.6-Flash: LMArena 1451, interval 1446–1456, 55 of 421 on LMArena, listed by LMArena as MiMo-V2.6-FlashInterval 1446–1456
- 18GPT-5.6 Terra · xhigh#56 of 421 on LMArenaGPT-5.6 Terra · xhigh: LMArena 1449, interval 1445–1453, 56 of 421 on LMArena, listed by LMArena as GPT-5.6 Terra (xHigh)Interval 1445–1453
- 19GPT-6.1 Sol · max#66 of 421 on LMArenaGPT-6.1 Sol · max: LMArena 1445, interval 1439–1451, 66 of 421 on LMArena, listed by LMArena as GPT-6.1 Sol (Max)Interval 1439–1451
- 20Step 5 Preview#70 of 421 on LMArenaStep 5 Preview: LMArena 1443, interval 1437–1450, 70 of 421 on LMArena, listed by LMArena as Step 5 PreviewInterval 1437–1450
- 21Inkling#71 of 421 on LMArenaInkling: LMArena 1443, interval 1439–1447, 71 of 421 on LMArena, listed by LMArena as InklingInterval 1439–1447
- 22Gemma 4 31B#73 of 421 on LMArenaGemma 4 31B: LMArena 1443, interval 1435–1450, 73 of 421 on LMArena, listed by LMArena as Gemma 4 31BInterval 1435–1450
- 23GPT-6 Astra · max#78 of 421 on LMArenaGPT-6 Astra · max: LMArena 1440, interval 1434–1445, 78 of 421 on LMArena, listed by LMArena as GPT-6 Astra (Max)Interval 1434–1445
- 24Qwen3.8 27B#85 of 421 on LMArenaQwen3.8 27B: LMArena 1437, interval 1432–1441, 85 of 421 on LMArena, listed by LMArena as Qwen3.8-27BInterval 1432–1441
- 25Gemini 3.5 Flash-Lite#90 of 421 on LMArenaGemini 3.5 Flash-Lite: LMArena 1435, interval 1430–1439, 90 of 421 on LMArena, listed by LMArena as Gemini 3.5 Flash-LiteInterval 1430–1439
- 26GPT-5.6 Luna · xhigh#94 of 421 on LMArenaGPT-5.6 Luna · xhigh: LMArena 1431, interval 1427–1435, 94 of 421 on LMArena, listed by LMArena as GPT-5.6 Luna (xHigh)Interval 1427–1435
- 27Grok 4.6 · high#96 of 421 on LMArenaGrok 4.6 · high: LMArena 1430, interval 1425–1435, 96 of 421 on LMArena, listed by LMArena as Grok 4.6 (High)Interval 1425–1435
- 28MiniMax-M3#100 of 421 on LMArenaMiniMax-M3: LMArena 1428, interval 1424–1432, 100 of 421 on LMArena, listed by LMArena as MiniMax-M3Interval 1424–1432
- 29Mistral Medium 3.5#114 of 421 on LMArenaMistral Medium 3.5: LMArena 1421, interval 1414–1427, 114 of 421 on LMArena, listed by LMArena as Mistral Medium 3.5Interval 1414–1427
- 30Inkling Small#136 of 421 on LMArenaInkling Small: LMArena 1414, interval 1410–1419, 136 of 421 on LMArena, listed by LMArena as Inkling-SmallInterval 1410–1419
- 31Mistral Large 4#141 of 421 on LMArenaMistral Large 4: LMArena 1411, interval 1403–1420, 141 of 421 on LMArena, listed by LMArena as Mistral Large 4Interval 1403–1420
- 32Inkling 2 Small#153 of 421 on LMArenaInkling 2 Small: LMArena 1403, interval 1391–1414, 153 of 421 on LMArena, listed by LMArena as Inkling 2 SmallInterval 1391–1414
- 33Grok 4.7 · xhigh#158 of 421 on LMArenaGrok 4.7 · xhigh: LMArena 1400, interval 1395–1405, 158 of 421 on LMArena, listed by LMArena as Grok 4.7 (xHigh)Interval 1395–1405
- 34Claude Haiku 5.5 · high#160 of 421 on LMArenaClaude Haiku 5.5 · high: LMArena 1398, interval 1392–1404, 160 of 421 on LMArena, listed by LMArena as Claude Haiku 5.5 (High)Interval 1392–1404
- 35GPT-6 Sol · max#163 of 421 on LMArenaGPT-6 Sol · max: LMArena 1397, interval 1391–1402, 163 of 421 on LMArena, listed by LMArena as GPT-6 Sol (Max)Interval 1391–1402
- 36Muse Glimmer#172 of 421 on LMArenaMuse Glimmer: LMArena 1389, interval 1379–1399, 172 of 421 on LMArena, listed by LMArena as Muse GlimmerInterval 1379–1399
- 37GPT-6 Luna · max#173 of 421 on LMArenaGPT-6 Luna · max: LMArena 1388, interval 1384–1393, 173 of 421 on LMArena, listed by LMArena as GPT-6 Luna (Max)Interval 1384–1393
- 38Solar Pro 4#174 of 421 on LMArenaSolar Pro 4: LMArena 1388, interval 1383–1393, 174 of 421 on LMArena, listed by LMArena as Solar Pro 4Interval 1383–1393
- 39gpt-oss-120b#197 of 421 on LMArenagpt-oss-120b: LMArena 1366, interval 1361–1370, 197 of 421 on LMArena, listed by LMArena as gpt-oss-120bInterval 1361–1370
- 40Command A+ · 03-2025#230 of 421 on LMArenaCommand A+ · 03-2025: LMArena 1331, interval 1327–1334, 230 of 421 on LMArena, listed by LMArena as Command A (03-2025)Interval 1327–1334
- 41Granite 4.2 8B#242 of 421 on LMArenaGranite 4.2 8B: LMArena 1319, interval 1307–1330, 242 of 421 on LMArena, listed by LMArena as Granite 4.2 8BInterval 1307–1330
- 42Granite 4.2 3B#258 of 421 on LMArenaGranite 4.2 3B: LMArena 1292, interval 1281–1304, 258 of 421 on LMArena, listed by LMArena as Granite 4.2 3BInterval 1281–1304
379 more LMArena entries, under LMArena’s own names
- #9 Claude Opus 4.6 (High) 1501
- #11 Claude Opus 4.6 1497
- #12 Claude Fable 5 (High) 1492
- #15 Claude Opus 4.7 (High) 1490
- #16 Gemini 3.7 Flash (High) 1489
- #17 Qwen3.8-Max 1485
- #18 Muse Spark 1.2 (xHigh) 1483
- #19 Claude Opus 4.7 1483
- #20 Gemini 3.5 Flash (High) 1483
- #21 Gemini 3.1 Pro Preview 1480
- #22 Gemini 3 Pro 1480
- #23 Gemini 3.6 Flash (High) 1479
- #24 Muse Spark 1.1 1479
- #25 Gemini 3.5 Flash (Medium) 1478
- #27 Qwen3.7-Max-Preview 1475
- #29 Muse Spark 1474
- #31 GPT-5.5 (High) 1471
- #32 Qwen3.5-Max-Preview 1471
- #33 GLM-5.2 (Max) 1469
- #35 GPT-5.4 (High) 1469
- #36 ERNIE 5.1 1468
- #37 GPT-5.5 1466
- #38 Gemini 3 Flash 1466
- #39 MiMo-V2.5-Pro 1464
- #40 DeepSeek-V4.1-Flash-20260910 (Max) 1464
- #41 GLM-5.1 1462
- #42 Claude Opus 4.8 (High) 1462
- #44 Claude Sonnet 4.6 1458
- #45 Gemini 2.5 Pro 1458
- #46 GPT-5.6 Sol (xHigh) 1457
- #47 Kimi K2.6 1455
- #48 Qwen3.7-Plus 1454
- #49 Claude Opus 4.8 1454
- #50 GPT-5.4 1453
- #51 DeepSeek-V4-Pro 1451
- #52 Grok 4.20 Beta (Reasoning, 0309) 1451
- #53 Grok 4.20 Multi-Agent Beta (0309) 1451
- #54 Claude Opus 4.5 (20251101) 1451
- #57 amazon-nova-experimental-chat (26-02-10) 1449
- #58 Dola Seed 2.0 Pro 1448
- #59 Claude Opus 4.5 (High, 32K, 20251101) 1448
- #60 Grok 4.5 1448
- #61 Qwen3.6-Max-Preview 1446
- #62 GLM-5 1446
- #63 NVIDIA-Nemotron-3-Ultra-550B-A55B (NVFP4) 1445
- #64 DeepSeek-V4-Pro Preview (High) 1445
- #65 DeepSeek-V4-Pro (High, 20260813) 1445
- #67 Kimi K2.5 (Thinking) 1445
- #68 Grok 4.20 Beta 1 1444
- #69 ERNIE 5.0 (0110) 1444
- #72 ERNIE-5.0-Preview (1203) 1443
- #74 GPT-5.1 (High) 1443
- #75 Gemini 3 Flash (Minimal) 1442
- #76 Claude Sonnet 5 (High) 1442
- #77 GLM-4.6 1440
- #79 Qwen3-Max-Preview 1439
- #80 Hy3 1439
- #81 Claude Sonnet 4.5 (20250929) 1439
- #82 GPT-5.2 Chat (20260210) 1438
- #83 Qwen3.5-397B-A17B 1438
- #84 Grok 4.1 (Thinking) 1437
- #86 MiMo-V2-Pro 1436
- #87 Grok 4.1 1436
- #88 Qwen3.6-Plus 1436
- #89 GLM-4.7 1435
- #91 Claude Sonnet 4.5 (High, 32K, 20250929) 1434
- #92 Gemma 4 26B A4B 1434
- #93 DeepSeek-V4-Flash 1432
- #95 GLM-4.5 1430
- #97 ERNIE-5.0-Preview (1022) 1430
- #98 Mistral Large 3 1429
- #99 ChatGPT-4o Latest (20250326) 1429
- #101 DeepSeek-R1 (0528) 1428
- #102 MiMo-V2.5 1427
- #103 LongCat-Flash-Chat Exp (2602) 1426
- #104 Grok 3 Preview (02-24) 1426
- #105 DeepSeek-V3.2-Exp (Thinking) 1425
- #106 DeepSeek-V4-Flash Preview (High) 1425
- #107 Mistral Medium 3.1 (2508) 1425
- #108 DeepSeek-V3.2 1425
- #109 Beam 1.0 501B A23B 1424
- #110 MiMo-V2-Omni 1423
- #111 LongCat-Flash-Chat 1423
- #112 DeepSeek-V3.2-Exp 1422
- #113 GPT-5.1 1422
- #115 Kimi K2.5 (Instant) 1420
- #116 DeepSeek-V3.2 (Thinking) 1420
- #117 Qwen3-VL-235B-A22B-Instruct 1420
- #118 DeepSeek-V3.1 1420
- #119 Claude Opus 4.1 (Thinking 16K, 20250805) 1419
- #120 GPT-5.5 Instant 1419
- #121 amazon-nova-experimental-chat (12-10) 1419
- #122 DeepSeek-V3.1-Terminus (Thinking) 1419
- #123 Qwen3-235B-A22B-Instruct (2507) 1419
- #124 Claude Opus 4.1 (20250805) 1419
- #125 Qwen3.5-122B-A10B 1417
- #126 GPT-4.5 Preview (2025-02-27) 1417
- #127 DeepSeek-V3.1-Terminus 1417
- #128 Qwen3-Next-80B-A3B-Instruct 1417
- #129 Gemini 2.5 Flash 1417
- #130 DeepSeek-V3.1 (Thinking) 1417
- #131 GPT-5.2 (High) 1416
- #132 Gemini 3.1 Flash-Lite Preview 1416
- #133 Qwen3-235B-A22B-Thinking (2507) 1415
- #134 amazon-nova-experimental-chat (11-10) 1415
- #135 Kimi K2 Thinking Turbo 1414
- #137 Qwen3-Max (2025-09-23) 1413
- #138 GPT-5.2 1412
- #139 GPT-5.4 mini (High) 1412
- #140 Grok 4 (0709) 1412
- #142 MiMo-V2-Flash (Non-Thinking) 1411
- #143 o3 (2025-04-16) 1410
- #144 Qwen3.5-27B 1409
- #145 Grok 4.1 Fast (Reasoning) 1408
- #146 NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4) 1407
- #147 Grok 4 Fast (Non-Reasoning) 1407
- #148 Gemini 2.5 Flash Preview (09-2025) 1407
- #149 GPT-5 (High) 1406
- #150 Hunyuan Vision 1.5 Thinking 1406
- #151 MiniMax-M2.7 1405
- #152 GPT-5 Chat 1404
- #154 Step-3.5-Flash 1402
- #155 Hy3 preview 1401
- #156 Hunyuan T1 (20250711) 1401
- #157 Qwen3-VL-235B-A22B-Thinking 1401
- #159 Grok 4 Fast (Reasoning) 1400
- #161 Qwen3.5-Flash 1397
- #162 Grok 4.3 1397
- #164 Claude Haiku 4.5 (20251001) 1396
- #165 amazon-nova-experimental-chat (26-01-10) 1395
- #166 MiMo-V2-Flash (Thinking) 1395
- #167 Qwen3.5-35B-A3B 1395
- #168 amazon-nova-experimental-chat (10-20) 1394
- #169 Qwen3-235B-A22B (Non-Thinking) 1394
- #170 MiniMax-M2.1 Preview 1392
- #171 GPT-5.3 Chat 1389
- #175 Qwen3-30B-A3B-Instruct (2507) 1384
- #176 GLM-4.5-Air 1384
- #177 GPT-4.1 (2025-04-14) 1383
- #178 Kimi K2 Preview (0905) 1380
- #179 Gemini 2.5 Flash-Lite Preview (Non-Thinking, 09-2025) 1379
- #180 Hunyuan TurboS (20250416) 1377
- #181 Claude Opus 4 (Thinking 16K, 20250514) 1377
- #182 NVIDIA-Nemotron-3-Super-120B-A12B 1377
- #183 GLM-4.6V 1377
- #184 DeepSeek-V3 (0324) 1375
- #185 GPT-5 mini (High) 1373
- #186 DeepSeek-R1 1373
- #187 GPT-5.4 nano (High) 1372
- #188 Kimi K2 Preview (0711) 1371
- #189 Mistral Medium 3 (2505) 1370
- #190 Grok 3 Mini (High) 1370
- #191 Gemini 2.5 Flash-Lite Preview (Thinking, 06-17) 1369
- #192 Qwen3-Next-80B-A3B-Thinking 1368
- #193 Qwen2.5-Max 1367
- #194 Qwen3-235B-A22B 1366
- #195 Claude Opus 4 (20250514) 1366
- #196 o1 (2024-12-17) 1366
- #198 Grok 3 Mini Beta 1364
- #199 amazon-nova-experimental-chat (10-09) 1364
- #200 Amazon Nova 2 Lite 1362
- #201 Granite 4.2 30B 1362
- #202 Ling-flash-2.0 1361
- #203 MiniMax-M2.5 1359
- #204 Gemma 3 27B IT 1358
- #205 Qwen3-Coder-480B-A35B-Instruct 1357
- #206 INTELLECT-3 1356
- #207 Mercury 2 1355
- #208 Gemini 2.0 Flash (001) 1354
- #209 o4-mini (2025-04-16) 1353
- #210 o1-preview 1353
- #211 Claude Sonnet 4 (Thinking 32K, 20250514) 1351
- #212 GLM-4.7-Flash 1351
- #213 Step-3 1350
- #214 NVIDIA-Nemotron-3-Nano-30B-A3B (BF16) 1348
- #215 MiniMax-M1 1343
- #216 GPT-4.1 mini (2025-04-14) 1341
- #217 MiniMax-M2 1340
- #218 Qwen3-32B 1340
- #219 Trinity Large Thinking 1340
- #220 Claude Sonnet 4 (20250514) 1339
- #221 Trinity Large Preview 1339
- #222 Llama 3.3 Nemotron Super 49B v1.5 1338
- #223 Mistral Small 3.2 (2506) 1338
- #224 o3-mini (High) 1337
- #225 Step-1o Turbo (202506) 1336
- #226 Gemma 3 12B IT 1334
- #227 Ring-flash-2.0 1334
- #228 GLM-4.5V 1333
- #229 DeepSeek-V3 1333
- #231 GLM-4-Plus (0111) 1331
- #232 Gemini 2.0 Flash-Lite Preview (02-05) 1330
- #233 NVIDIA-Nemotron-3.5-Lightning-30B-A3B (NVFP4) 1329
- #234 QwQ-32B 1329
- #235 Qwen-Plus (0125) 1326
- #236 Step-2 16K Exp (202412) 1321
- #237 GPT-5 nano (High) 1320
- #238 Hunyuan TurboS (20250226) 1320
- #239 Llama 3.1 Nemotron Ultra 253B v1 1320
- #240 o3-mini 1320
- #241 Gemini 1.5 Pro (002) 1319
- #243 o1-mini 1317
- #244 Qwen3-30B-A3B 1317
- #245 Claude 3.7 Sonnet (Thinking 32K, 20250219) 1314
- #246 Hunyuan Turbo (0110) 1311
- #247 Olmo 3.1 32B Instruct 1311
- #248 Llama 3.3 Nemotron Super 49B v1 1308
- #249 Gemma 3n E4B IT 1306
- #250 Grok 2 (2024-08-13) 1305
- #251 Yi Lightning 1302
- #252 GPT-4o (2024-05-13) 1300
- #253 Claude 3.7 Sonnet (20250219) 1299
- #254 Qwen2.5-Plus (1127) 1299
- #255 Olmo 3 32B Think 1299
- #256 Claude 3.5 Sonnet (20241022) 1298
- #257 DeepSeek-V2.5 (1210) 1294
- #259 Athene V2 Chat 1292
- #260 Gemma 3 4B IT 1291
- #261 Granite 4.1 8B 1291
- #262 GLM-4-Plus 1290
- #263 Hunyuan Large (2025-02-10) 1288
- #264 gpt-oss-20b 1288
- #265 Llama 4 Maverick 17B 128E Instruct 1287
- #266 Molmo 2 8B 1287
- #267 Gemini 1.5 Flash (002) 1287
- #268 GPT-4o mini (2024-07-18) 1286
- #269 GPT-4.1 nano (2025-04-14) 1285
- #270 Llama 3.1 405B Instruct (BF16) 1284
- #271 Llama 3.1 Nemotron 70B Instruct 1283
- #272 GPT-4o (2024-08-06) 1283
- #273 Qwen-Max (0919) 1282
- #274 Llama 3.1 405B Instruct (FP8) 1282
- #275 Mercury 1282
- #276 Claude 3.5 Sonnet (20240620) 1281
- #277 Grok 2 mini (2024-08-13) 1281
- #278 Llama 4 Scout 17B 16E Instruct 1279
- #279 Gemini Advanced (0514) 1279
- #280 Mistral Small 3.1 24B Instruct (2503) 1277
- #281 Llama 3.3 70B Instruct 1274
- #282 Hunyuan Standard (2025-02-10) 1274
- #283 Gemini 1.5 Pro (001) 1274
- #284 GPT-4 Turbo (2024-04-09) 1272
- #285 DeepSeek-V2.5 1271
- #286 Olmo 3.1 32B Think 1271
- #287 Qwen2.5-72B-Instruct 1269
- #288 Mistral Large 2 (2407) 1266
- #289 Mistral Large (2411) 1265
- #290 Athene 70B (0725) 1265
- #291 GPT-4 Turbo Preview (1106) 1263
- #292 Hunyuan Large Vision 1263
- #293 GPT-4 Turbo Preview (0125) 1262
- #294 Claude 3 Opus (20240229) 1262
- #295 Llama 3.1 70B Instruct 1261
- #296 Amazon Nova Pro 1.0 1259
- #297 Llama 3.1 Tulu 3 70B 1256
- #298 Claude 3.5 Haiku (20241022) 1255
- #299 Magistral Medium (2506) 1255
- #300 Reka Core (20240904) 1248
- #301 Granite-4.0-H-Small 1241
- #302 Gemini 1.5 Flash (001) 1239
- #303 Jamba 1.5 Large 1237
- #304 Mistral Small 24B Instruct (2501) 1234
- #305 Gemma 2 27B IT 1231
- #306 Qwen2.5-Coder-32B-Instruct 1230
- #307 Command R+ (08-2024) 1229
- #308 Amazon Nova Lite 1.0 1229
- #309 Llama 3.1 Nemotron 51B Instruct 1228
- #310 Gemma 2 9B IT SimPO 1227
- #311 GLM-4 (0520) 1226
- #312 Gemini 1.5 Flash-8B (001) 1226
- #313 Nemotron-4 340B Instruct 1225
- #314 Aya Expanse 32B 1224
- #315 Llama 3 70B Instruct 1221
- #316 Claude 3 Sonnet (20240229) 1218
- #317 OLMo 2 32B Instruct (0325) 1218
- #318 Reka Flash (20240904) 1218
- #319 Phi-4 1217
- #320 Amazon Nova Micro 1.0 1208
- #321 Gemma 2 9B IT 1207
- #322 GPT-4 (0314) 1206
- #323 Command R+ 1204
- #324 Qwen2-72B-Instruct 1203
- #325 Hunyuan Standard 256K 1202
- #326 Claude 3 Haiku (20240307) 1195
- #327 Llama 3.1 Tulu 3 8B 1193
- #328 DeepSeek-Coder-V2 1191
- #329 Ministral 8B (2410) 1191
- #330 Command R (08-2024) 1187
- #331 Jamba 1.5 Mini 1187
- #332 Llama 3.1 8B Instruct 1187
- #333 GPT-4 (0613) 1186
- #334 Aya Expanse 8B 1185
- #335 Mistral Large (2402) 1177
- #336 Qwen1.5-110B-Chat 1175
- #337 Yi 1.5 34B Chat 1173
- #338 Reka Flash 21B (Online, 20240226) 1170
- #339 Qwen1.5-72B-Chat 1166
- #340 Llama 3 8B Instruct 1166
- #341 Mistral Medium 1165
- #342 Reka Flash 21B (20240226) 1165
- #343 Command R 1163
- #344 Mixtral 8x22B Instruct v0.1 1162
- #345 QwQ-32B-Preview 1162
- #346 InternLM2.5 20B Chat 1158
- #347 Gemma 2 2B IT 1156
- #348 Granite 3.1 8B Instruct 1150
- #349 Gemini Pro (Dev API) 1149
- #350 Zephyr ORPO 141B A35B v0.1 1144
- #351 Phi-3 Medium 4K Instruct 1138
- #352 Qwen1.5-32B-Chat 1137
- #353 Starling LM 7B Beta 1132
- #354 Mixtral 8x7B Instruct v0.1 1132
- #355 Gemini Pro 1131
- #356 Yi 34B Chat 1129
- #357 Qwen1.5-14B-Chat 1128
- #358 Granite 3.1 2B Instruct 1127
- #359 GPT-3.5 Turbo (0125) 1125
- #360 Tulu 2 DPO 70B 1121
- #361 WizardLM 70B 1120
- #362 DBRX Instruct Preview 1119
- #363 Llama 2 70B Chat 1115
- #364 Nous Hermes 2 Mixtral 8x7B DPO 1112
- #365 Phi-3 Small 8K Instruct 1110
- #366 Llama 3.2 3B Instruct 1110
- #367 Starling LM 7B Alpha 1108
- #368 OpenChat 3.5 (0106) 1107
- #369 Vicuna 33B 1105
- #370 DeepSeek LLM 67B Chat 1105
- #371 Snowflake Arctic Instruct 1101
- #372 Llama 2 70B SteerLM Chat 1098
- #373 OpenChat 3.5 1097
- #374 Granite 3.0 8B Instruct 1096
- #375 GPT-3.5 Turbo (1106) 1094
- #376 Gemma 1.1 7B IT 1094
- #377 OpenHermes 2.5 Mistral 7B 1093
- #378 Mistral 7B Instruct v0.2 1090
- #379 Llama 2 13B Chat 1085
- #380 Qwen1.5-7B-Chat 1084
- #381 SOLAR 10.7B Instruct v1.0 1083
- #382 Dolphin 2.2.1 Mistral 7B 1081
- #383 Phi-3 Mini 4K Instruct (June 2024) 1080
- #384 Granite 3.0 2B Instruct 1080
- #385 WizardLM 13B 1077
- #386 Phi-3 Mini 4K Instruct 1073
- #387 Zephyr 7B Beta 1069
- #388 MPT 30B Chat 1069
- #389 Code Llama 34B Instruct 1066
- #390 Zephyr 7B Alpha 1058
- #391 Vicuna 13B 1058
- #392 Code Llama 70B Instruct 1057
- #393 Gemma 7B IT 1056
- #394 Llama 3.2 1B Instruct 1055
- #395 Falcon 180B Chat 1054
- #396 Llama 2 7B Chat 1053
- #397 Guanaco 33B 1053
- #398 Qwen-14B-Chat 1051
- #399 Phi-3 Mini 128K Instruct 1050
- #400 SmolLM2 1.7B Instruct 1042
- #401 StripedHyena Nous 7B 1038
- #402 OLMo 7B Instruct 1032
- #403 Vicuna 7B 1031
- #404 PaLM 2 1027
- #405 Mistral 7B Instruct 1024
- #406 Gemma 1.1 2B IT 1022
- #407 Gemma 2B IT 1002
- #408 Qwen1.5-4B-Chat 997
- #409 Koala 13B 990
- #410 ChatGLM3 6B 972
- #411 GPT4All 13B Snoozy 956
- #412 MPT 7B Chat 955
- #413 RWKV-4 Raven 14B 948
- #414 ChatGLM2 6B 939
- #415 Alpaca 13B 933
- #416 ChatGLM 6B 919
- #417 OpenAssistant Pythia 12B 916
- #418 FastChat-T5 3B 894
- #419 StableLM Tuned Alpha 7B 867
- #420 Dolly 2.0 12B 851
- #421 LLaMA 13B 834
The score is LMArena’s Arena score on its overall text leaderboard, a rating fitted to every head-to-head vote, with the interval LMArena publishes beside it. A higher score means people picked that model’s replies more often.
42 of LMArena’s 421 entries are models this site shows, ranked above under the site’s names. The other 379 are listed under LMArena’s own names.
Data: LMArena’s text leaderboard, from its open dataset, under CC BY 4.0, data of . Part of External Composite and Everything Composite. About this source
FrontierMath measures hard mathematics. Its problems were written by mathematicians, run from difficult undergraduate problems to ones an advanced graduate student would find exploratory, and have never been published, so no model can have read them; each answer is checked by computer. Neither Artificial Analysis’s index nor the Gauntlet sets research-grade mathematics.
FrontierMath · FrontierMath tiers 1–3, share solved · data of · CC BY 4.0
Claude Fable 5.2 · max makes 1.3× fewer misses than GPT-6 Astra · max.
- 1Claude Fable 5.2 · maxClaude Fable 5.2 · max: FrontierMath tiers 1–3, share solved 95%, the leader
- 2GPT-6 Astra · maxGPT-6 Astra · max: FrontierMath tiers 1–3, share solved 94%, 1.3 times the misses of the model above
- 3GPT-6.1 Sol · maxGPT-6.1 Sol · max: FrontierMath tiers 1–3, share solved 94%, 1.0 times the misses of the model above
- 4Claude Opus 5.5 · maxClaude Opus 5.5 · max: FrontierMath tiers 1–3, share solved 91%, 1.4 times the misses of the model above
- 5GPT-6.1 Terra · maxGPT-6.1 Terra · max: FrontierMath tiers 1–3, share solved 91%, 1.0 times the misses of the model above
- 6Gemini 4 Carbon · highGemini 4 Carbon · high: FrontierMath tiers 1–3, share solved 91%, 1.0 times the misses of the model above
- 7Claude Fable 5.1 · maxClaude Fable 5.1 · max: FrontierMath tiers 1–3, share solved 90%, 1.0 times the misses of the model above
- 8GPT-6 Sol · maxGPT-6 Sol · max: FrontierMath tiers 1–3, share solved 90%, 1.0 times the misses of the model above
- 9Claude Sonnet 5.5 · maxClaude Sonnet 5.5 · max: FrontierMath tiers 1–3, share solved 89%, 1.1 times the misses of the model above
- 10Gemini 4 Argon · highGemini 4 Argon · high: FrontierMath tiers 1–3, share solved 86%, 1.2 times the misses of the model above
- 11GPT-5.6 Terra · maxGPT-5.6 Terra · max: FrontierMath tiers 1–3, share solved 86%, 1.0 times the misses of the model above
- 12Claude Opus 5 · maxClaude Opus 5 · max: FrontierMath tiers 1–3, share solved 86%, 1.0 times the misses of the model above
- 13GPT-5.6 Luna · maxGPT-5.6 Luna · max: FrontierMath tiers 1–3, share solved 82%, 1.2 times the misses of the model above
- 14Muse Wonder · maxMuse Wonder · max: FrontierMath tiers 1–3, share solved 82%, 1.0 times the misses of the model above
- 15GPT-6 Luna · maxGPT-6 Luna · max: FrontierMath tiers 1–3, share solved 79%, 1.2 times the misses of the model above
- 16Kimi K3.1 Pro · maxKimi K3.1 Pro · max: FrontierMath tiers 1–3, share solved 78%, 1.0 times the misses of the model above
- 17Claude Haiku 5.5 · maxClaude Haiku 5.5 · max: FrontierMath tiers 1–3, share solved 75%, 1.1 times the misses of the model above
- 18Muse Spark 1.3 · xhighMuse Spark 1.3 · xhigh: FrontierMath tiers 1–3, share solved 74%, 1.0 times the misses of the model above
- 19Muse Spark 1.3 · maxMuse Spark 1.3 · max: FrontierMath tiers 1–3, share solved 74%, 1.0 times the misses of the model above
- 20Kimi K3 · maxKimi K3 · max: FrontierMath tiers 1–3, share solved 72%, 1.1 times the misses of the model above
- 21DeepSeek V4.1 Flash 1017 · maxDeepSeek V4.1 Flash 1017 · max: FrontierMath tiers 1–3, share solved 71%, 1.0 times the misses of the model above
- 22Muse Spark Open · xhighMuse Spark Open · xhigh: FrontierMath tiers 1–3, share solved 70%, 1.0 times the misses of the model above
- 23GLM-5.3 · maxGLM-5.3 · max: FrontierMath tiers 1–3, share solved 69%, 1.0 times the misses of the model above
- 24Gemini 3.8 Flash · highGemini 3.8 Flash · high: FrontierMath tiers 1–3, share solved 68%, 1.0 times the misses of the model above
- 25Grok 4.6 · xhighGrok 4.6 · xhigh: FrontierMath tiers 1–3, share solved 66%, 1.1 times the misses of the model above
- 26Qwen3.8 Max (0902) · xhighQwen3.8 Max (0902) · xhigh: FrontierMath tiers 1–3, share solved 66%, 1.0 times the misses of the model above
- 27DeepSeek V4 Pro 0813 · maxDeepSeek V4 Pro 0813 · max: FrontierMath tiers 1–3, share solved 65%, 1.0 times the misses of the model above
- 28Mistral Large 4 · highMistral Large 4 · high: FrontierMath tiers 1–3, share solved 59%, 1.2 times the misses of the model above
- 29GLM-5.3-Flash · maxGLM-5.3-Flash · max: FrontierMath tiers 1–3, share solved 56%, 1.1 times the misses of the model above
- 30Grok 4.7 · xhighGrok 4.7 · xhigh: FrontierMath tiers 1–3, share solved 53%, 1.1 times the misses of the model above
- 31Inkling Small · xhighInkling Small · xhigh: FrontierMath tiers 1–3, share solved 46%, 1.1 times the misses of the model above
- 32GPT-5.6 Luna · lowGPT-5.6 Luna · low: FrontierMath tiers 1–3, share solved 41%, 1.1 times the misses of the model above
- 33GPT-5.6 Luna · noneGPT-5.6 Luna · none: FrontierMath tiers 1–3, share solved 40%, 1.0 times the misses of the model above
- 34Inkling · xhighInkling · xhigh: FrontierMath tiers 1–3, share solved 33%, 1.1 times the misses of the model above
- 35Gemini 3.1 Flash-Lite · highGemini 3.1 Flash-Lite · high: FrontierMath tiers 1–3, share solved 28%, 1.1 times the misses of the model above
- 36Gemini 3.5 Flash-Lite · highGemini 3.5 Flash-Lite · high: FrontierMath tiers 1–3, share solved 26%, 1.0 times the misses of the model above
- 37Gemini 3.1 Flash-Lite · lowGemini 3.1 Flash-Lite · low: FrontierMath tiers 1–3, share solved 23%, 1.0 times the misses of the model above
- 38Gemini 3.1 Flash-Lite · minimalGemini 3.1 Flash-Lite · minimal: FrontierMath tiers 1–3, share solved 21%, 1.0 times the misses of the model above
Placed by gap: equal distances are equal ratios of misses, across only the models shown.
- 1Claude Fable 5.2 · max#1 of 125 on FrontierMathClaude Fable 5.2 · max: FrontierMath 95.1%, interval 93%–98%, 1 of 125 on FrontierMath, listed by FrontierMath as claude-fable-5-2_maxInterval 93%–98%
- 2GPT-6 Astra · max#2 of 125 on FrontierMathGPT-6 Astra · max: FrontierMath 93.7%, interval 91%–97%, 2 of 125 on FrontierMath, listed by FrontierMath as gpt-6-astra_maxInterval 91%–97%
- 3GPT-6.1 Sol · max#3 of 125 on FrontierMathGPT-6.1 Sol · max: FrontierMath 93.7%, interval 91%–97%, 3 of 125 on FrontierMath, listed by FrontierMath as gpt-6.1-sol_maxInterval 91%–97%
- 4Claude Opus 5.5 · max#4 of 125 on FrontierMathClaude Opus 5.5 · max: FrontierMath 91.2%, interval 88%–95%, 4 of 125 on FrontierMath, listed by FrontierMath as claude-opus-5-5_maxInterval 88%–95%
- 5GPT-6.1 Terra · max#5 of 125 on FrontierMathGPT-6.1 Terra · max: FrontierMath 90.9%, interval 88%–94%, 5 of 125 on FrontierMath, listed by FrontierMath as gpt-6.1-terra_maxInterval 88%–94%
- 6Gemini 4 Carbon · high#6 of 125 on FrontierMathGemini 4 Carbon · high: FrontierMath 90.5%, interval 87%–94%, 6 of 125 on FrontierMath, listed by FrontierMath as gemini-4-carbon_highInterval 87%–94%
- 7Claude Fable 5.1 · max#7 of 125 on FrontierMathClaude Fable 5.1 · max: FrontierMath 90.2%, interval 87%–94%, 7 of 125 on FrontierMath, listed by FrontierMath as claude-fable-5-1_maxInterval 87%–94%
- 8GPT-6 Sol · max#8 of 125 on FrontierMathGPT-6 Sol · max: FrontierMath 89.8%, interval 86%–93%, 8 of 125 on FrontierMath, listed by FrontierMath as gpt-6-sol_maxInterval 86%–93%
- 9Claude Sonnet 5.5 · max#10 of 125 on FrontierMathClaude Sonnet 5.5 · max: FrontierMath 88.8%, interval 85%–93%, 10 of 125 on FrontierMath, listed by FrontierMath as claude-sonnet-5-5_maxInterval 85%–93%
- 10Gemini 4 Argon · high#13 of 125 on FrontierMathGemini 4 Argon · high: FrontierMath 86.0%, interval 82%–90%, 13 of 125 on FrontierMath, listed by FrontierMath as gemini-4-argon_highInterval 82%–90%
- 11GPT-5.6 Terra · max#14 of 125 on FrontierMathGPT-5.6 Terra · max: FrontierMath 86.0%, interval 82%–90%, 14 of 125 on FrontierMath, listed by FrontierMath as gpt-5.6-terra_maxInterval 82%–90%
- 12Claude Opus 5 · max#15 of 125 on FrontierMathClaude Opus 5 · max: FrontierMath 85.6%, interval 82%–90%, 15 of 125 on FrontierMath, listed by FrontierMath as claude-opus-5_maxInterval 82%–90%
- 13GPT-5.6 Luna · max#18 of 125 on FrontierMathGPT-5.6 Luna · max: FrontierMath 82.1%, interval 78%–87%, 18 of 125 on FrontierMath, listed by FrontierMath as gpt-5.6-luna_maxInterval 78%–87%
- 14Muse Wonder · max#19 of 125 on FrontierMathMuse Wonder · max: FrontierMath 82.1%, interval 78%–87%, 19 of 125 on FrontierMath, listed by FrontierMath as muse-wonder-1.0_maxInterval 78%–87%
- 15GPT-6 Luna · max#21 of 125 on FrontierMathGPT-6 Luna · max: FrontierMath 78.9%, interval 74%–84%, 21 of 125 on FrontierMath, listed by FrontierMath as gpt-6-luna_maxInterval 74%–84%
- 16Kimi K3.1 Pro · max#23 of 125 on FrontierMathKimi K3.1 Pro · max: FrontierMath 78.2%, interval 73%–83%, 23 of 125 on FrontierMath, listed by FrontierMath as kimi-k3.1-pro-together_maxInterval 73%–83%
- 17Claude Haiku 5.5 · max#24 of 125 on FrontierMathClaude Haiku 5.5 · max: FrontierMath 75.1%, interval 70%–80%, 24 of 125 on FrontierMath, listed by FrontierMath as claude-haiku-5-5_maxInterval 70%–80%
- 18Muse Spark 1.3 · xhigh#26 of 125 on FrontierMathMuse Spark 1.3 · xhigh: FrontierMath 74.4%, interval 69%–80%, 26 of 125 on FrontierMath, listed by FrontierMath as muse-spark-1.3_xhighInterval 69%–80%
- 19Muse Spark 1.3 · max#28 of 125 on FrontierMathMuse Spark 1.3 · max: FrontierMath 74.0%, interval 69%–79%, 28 of 125 on FrontierMath, listed by FrontierMath as muse-spark-1.3_maxInterval 69%–79%
- 20Kimi K3 · max#29 of 125 on FrontierMathKimi K3 · max: FrontierMath 72.2%, interval 67%–77%, 29 of 125 on FrontierMath, listed by FrontierMath as kimi-k3_maxInterval 67%–77%
- 21DeepSeek V4.1 Flash 1017 · max#31 of 125 on FrontierMathDeepSeek V4.1 Flash 1017 · max: FrontierMath 71.2%, interval 66%–77%, 31 of 125 on FrontierMath, listed by FrontierMath as deepseek-v4.1-flash-1017_maxInterval 66%–77%
- 22Muse Spark Open · xhigh#33 of 125 on FrontierMathMuse Spark Open · xhigh: FrontierMath 69.8%, interval 65%–75%, 33 of 125 on FrontierMath, listed by FrontierMath as Muse-Spark-Open-816B-A42B_xhighInterval 65%–75%
- 23GLM-5.3 · max#34 of 125 on FrontierMathGLM-5.3 · max: FrontierMath 68.8%, interval 63%–74%, 34 of 125 on FrontierMath, listed by FrontierMath as glm-5.3_maxInterval 63%–74%
- 24Gemini 3.8 Flash · high#35 of 125 on FrontierMathGemini 3.8 Flash · high: FrontierMath 68.4%, interval 63%–74%, 35 of 125 on FrontierMath, listed by FrontierMath as gemini-3.8-flash_highInterval 63%–74%
- 25Grok 4.6 · xhigh#39 of 125 on FrontierMathGrok 4.6 · xhigh: FrontierMath 66.0%, interval 61%–72%, 39 of 125 on FrontierMath, listed by FrontierMath as grok-4.6_xhighInterval 61%–72%
- 26Qwen3.8 Max (0902) · xhigh#41 of 125 on FrontierMathQwen3.8 Max (0902) · xhigh: FrontierMath 65.6%, interval 60%–71%, 41 of 125 on FrontierMath, listed by FrontierMath as qwen3.8-max-0902_xhighInterval 60%–71%
- 27DeepSeek V4 Pro 0813 · max#42 of 125 on FrontierMathDeepSeek V4 Pro 0813 · max: FrontierMath 64.6%, interval 59%–70%, 42 of 125 on FrontierMath, listed by FrontierMath as deepseek-v4-pro-0813_maxInterval 59%–70%
- 28Mistral Large 4 · high#48 of 125 on FrontierMathMistral Large 4 · high: FrontierMath 58.6%, interval 53%–64%, 48 of 125 on FrontierMath, listed by FrontierMath as mistral-large-4-0_highInterval 53%–64%
- 29GLM-5.3-Flash · max#52 of 125 on FrontierMathGLM-5.3-Flash · max: FrontierMath 55.8%, interval 50%–62%, 52 of 125 on FrontierMath, listed by FrontierMath as glm-5.3-flash_maxInterval 50%–62%
- 30Grok 4.7 · xhigh#57 of 125 on FrontierMathGrok 4.7 · xhigh: FrontierMath 53.0%, interval 47%–59%, 57 of 125 on FrontierMath, listed by FrontierMath as grok-4.7_xhighInterval 47%–59%
- 31Inkling Small · xhigh#61 of 125 on FrontierMathInkling Small · xhigh: FrontierMath 46.3%, interval 41%–52%, 61 of 125 on FrontierMath, listed by FrontierMath as inkling-small_xhighInterval 41%–52%
- 32GPT-5.6 Luna · low#67 of 125 on FrontierMathGPT-5.6 Luna · low: FrontierMath 41.4%, interval 36%–47%, 67 of 125 on FrontierMath, listed by FrontierMath as gpt-5.6-luna_lowInterval 36%–47%
- 33GPT-5.6 Luna · none#68 of 125 on FrontierMathGPT-5.6 Luna · none: FrontierMath 39.6%, interval 34%–45%, 68 of 125 on FrontierMath, listed by FrontierMath as gpt-5.6-luna_noneInterval 34%–45%
- 34Inkling · xhigh#77 of 125 on FrontierMathInkling · xhigh: FrontierMath 33.3%, interval 28%–39%, 77 of 125 on FrontierMath, listed by FrontierMath as Inkling_xhighInterval 28%–39%
- 35Gemini 3.1 Flash-Lite · high#84 of 125 on FrontierMathGemini 3.1 Flash-Lite · high: FrontierMath 27.7%, interval 23%–33%, 84 of 125 on FrontierMath, listed by FrontierMath as gemini-3.1-flash-lite_highInterval 23%–33%
- 36Gemini 3.5 Flash-Lite · high#86 of 125 on FrontierMathGemini 3.5 Flash-Lite · high: FrontierMath 26.0%, interval 21%–31%, 86 of 125 on FrontierMath, listed by FrontierMath as gemini-3.5-flash-lite_highInterval 21%–31%
- 37Gemini 3.1 Flash-Lite · low#91 of 125 on FrontierMathGemini 3.1 Flash-Lite · low: FrontierMath 22.5%, interval 18%–27%, 91 of 125 on FrontierMath, listed by FrontierMath as gemini-3.1-flash-lite_lowInterval 18%–27%
- 38Gemini 3.1 Flash-Lite · minimal#93 of 125 on FrontierMathGemini 3.1 Flash-Lite · minimal: FrontierMath 21.4%, interval 17%–26%, 93 of 125 on FrontierMath, listed by FrontierMath as gemini-3.1-flash-lite_minimalInterval 17%–26%
87 more FrontierMath entries, under FrontierMath’s own names
- #9 gpt-5.6-sol_max 89.1%
- #11 gpt-5.5-pro_xhigh 87.7%
- #12 claude-fable-5_max 87.0%
- #16 gpt-5.5_xhigh 85.3%
- #17 gpt-5.4-pro-2026-03-05_xhigh 82.5%
- #20 claude-opus-4-8_max 80.0%
- #22 gpt-5.4-2026-03-05_xhigh 78.6%
- #25 qwen3.8-max_xhigh 74.7%
- #27 gpt-5.2-pro-2025-12-11_xhigh 74.0%
- #30 gemini-3.7-flash_high 71.6%
- #32 claude-opus-4-7_max 70.2%
- #36 deepseek-v4.1-flash_max 67.4%
- #37 gpt-5.2-2025-12-11_xhigh 67.4%
- #38 claude-opus-4-6_max 66.0%
- #40 claude-sonnet-5_max 65.6%
- #43 qwen3.7-max 64.6%
- #44 gemini-3.5-flash_high 62.8%
- #45 gemini-3.1-pro-preview 59.6%
- #46 glm-5.2_max 59.2%
- #47 gemini-3.6-flash_high 58.9%
- #49 deepseek-v4-flash-0731_max 57.5%
- #50 grok-4.5_high 57.2%
- #51 kimi-k2.6 57.2%
- #53 gpt-5-pro-2025-10-06_high 55.8%
- #54 gpt-5-2025-08-07_high 55.4%
- #55 glm-5.2_low 54.7%
- #56 kimi-k2.7-code 54.0%
- #58 gemini-3-flash-preview 51.2%
- #59 gpt-5.4-mini-2026-03-17_xhigh 51.2%
- #60 gpt-5-mini-2025-08-07_high 46.7%
- #62 deepseek-v4-pro_max 45.3%
- #63 gpt-5.4-nano-2026-03-17_high 44.9%
- #64 grok-4.20-0309-reasoning 44.9%
- #65 grok-4.3_high 42.8%
- #66 glm-5.2_none 42.5%
- #69 qwen3.6-plus 38.2%
- #70 gpt-5-2025-08-07_low 37.2%
- #71 glm-5.1 36.8%
- #72 o4-mini-2025-04-16_high 36.1%
- #73 qwen3.6-27b 35.1%
- #74 claude-opus-4-5-20251101_32K 34.4%
- #75 qwen3.7-plus_none 34.4%
- #76 qwen3.6-27b_none 34.0%
- #78 o3-2025-04-16_high 33.3%
- #79 qwen3.6-plus_none 32.3%
- #80 qwen3.5-397b-a17b_none 31.2%
- #81 o3-2025-04-16_medium 29.8%
- #82 qwen3.5-397b-a17b 29.5%
- #83 o4-mini-2025-04-16_medium 28.8%
- #85 gpt-5.5-instant 26.3%
- #87 glm-5.1_none 24.9%
- #88 gemini-2.5-pro 24.6%
- #89 gpt-5.4-mini-2026-03-17_low 24.6%
- #90 claude-sonnet-4-5-20250929_32K 23.9%
- #92 qwen3.6-flash_none 22.5%
- #94 gpt-5.4-nano-2026-03-17_low 20.4%
- #95 qwen3.6-35b-a3b_none 20.4%
- #96 gpt-5-nano-2025-08-07_high 20.0%
- #97 o3-2025-04-16_low 19.3%
- #98 qwen3.7-flash 19.3%
- #99 qwen3.7-flash_none 19.3%
- #100 qwen3-max-2025-09-23 18.9%
- #101 o3-mini-2025-01-31_high 18.6%
- #102 gpt-5-2025-08-07_minimal 18.2%
- #103 gpt-5-mini-2025-08-07_low 18.2%
- #104 qwen3.5-flash_none 18.2%
- #105 qwen3.6-35b-a3b 17.5%
- #106 gpt-5.4-mini-2026-03-17_none 17.2%
- #107 qwen3.6-flash 17.2%
- #108 o4-mini-2025-04-16_low 16.1%
- #109 o1-2024-12-17_high 14.7%
- #110 claude-opus-4-1-20250805_32K 12.6%
- #111 o3-mini-2025-01-31_medium 10.5%
- #112 o1-2024-12-17_medium 10.2%
- #113 qwen3.5-flash 9.5%
- #114 o1-2024-12-17_low 8.4%
- #115 gpt-4.1-mini-2025-04-14 6.7%
- #116 gpt-4.1-2025-04-14 6.0%
- #117 gpt-5-mini-2025-08-07_minimal 6.0%
- #118 gpt-5-nano-2025-08-07_low 6.0%
- #119 gpt-5.4-nano-2026-03-17_none 4.6%
- #120 o3-mini-2025-01-31_low 3.9%
- #121 gpt-5-nano-2025-08-07_minimal 1.8%
- #122 gpt-4-turbo-2024-04-09 0.7%
- #123 gpt-4o-mini-2024-07-18 0.7%
- #124 gpt-4o-2024-08-06 0.4%
- #125 gpt-3.5-turbo-0125 0.0%
The score is the share of tiers 1 to 3 that a model at that setting solved in Epoch AI’s runs, with a 95% interval from Epoch AI’s standard error.
38 of FrontierMath’s 125 entries are models this site shows, ranked above under the site’s names. The other 87 are listed under FrontierMath’s own names.
Data: FrontierMath by Epoch AI, from Epoch AI’s open benchmark data, under CC BY 4.0, data of . Part of External Composite and Everything Composite. About this source
Vending-Bench 2 measures whether a model stays on course over a very long task. The model runs a simulated vending-machine business for a simulated year from a $500 start, finding suppliers, ordering stock, setting prices and handling refunds, and is scored on the money in its bank account at the end. Neither Artificial Analysis nor the Gauntlet runs one task anywhere near that long.
Vending-Bench 2 · Vending-Bench 2, money at the end of the year · data of · CC BY 4.0
Gemini 4 Carbon leads GPT-6 Astra by $970.
- 1Gemini 4 CarbonGemini 4 Carbon: Vending-Bench 2, money at the end of the year $16.5k, the leader
- 2GPT-6 AstraGPT-6 Astra: Vending-Bench 2, money at the end of the year $15.5k, $970 below the model above
- 3GPT-6 SolGPT-6 Sol: Vending-Bench 2, money at the end of the year $14.4k, $1,087 below the model above
- 4Gemini 4 ArgonGemini 4 Argon: Vending-Bench 2, money at the end of the year $13.7k, $710 below the model above
- 5GPT-6.1 SolGPT-6.1 Sol: Vending-Bench 2, money at the end of the year $12.9k, $858 below the model above
- 6Claude Opus 5Claude Opus 5: Vending-Bench 2, money at the end of the year $11.2k, $1,679 below the model above
- 7Claude Sonnet 5.5Claude Sonnet 5.5: Vending-Bench 2, money at the end of the year $11.2k, $17 below the model above
- 8Grok 4.7Grok 4.7: Vending-Bench 2, money at the end of the year $10.5k, $628 below the model above
- 9Muse WonderMuse Wonder: Vending-Bench 2, money at the end of the year $9.4k, $1,105 below the model above
- 10Claude Opus 5.5Claude Opus 5.5: Vending-Bench 2, money at the end of the year $9.2k, $197 below the model above
- 11GPT-6.1 TerraGPT-6.1 Terra: Vending-Bench 2, money at the end of the year $9.1k, $128 below the model above
- 12Grok 4.6Grok 4.6: Vending-Bench 2, money at the end of the year $9.0k, $60 below the model above
- 13GLM-5.3GLM-5.3: Vending-Bench 2, money at the end of the year $8.2k, $883 below the model above
- 14GPT-5.6 TerraGPT-5.6 Terra: Vending-Bench 2, money at the end of the year $7.3k, $820 below the model above
- 15Claude Fable 5.1Claude Fable 5.1: Vending-Bench 2, money at the end of the year $5.4k, $1,922 below the model above
- 16Kimi K3Kimi K3: Vending-Bench 2, money at the end of the year $5.2k, $257 below the model above
- 17Gemini 3.8 FlashGemini 3.8 Flash: Vending-Bench 2, money at the end of the year $5.1k, $71 below the model above
- 18GPT-5.6 LunaGPT-5.6 Luna: Vending-Bench 2, money at the end of the year $4.1k, $999 below the model above
- 19Claude Haiku 5.5Claude Haiku 5.5: Vending-Bench 2, money at the end of the year $3.6k, $492 below the model above
- 20MiniMax-M3MiniMax-M3: Vending-Bench 2, money at the end of the year $2.2k, $1,445 below the model above
- 21gpt-oss-120bgpt-oss-120b: Vending-Bench 2, money at the end of the year −$22, $2,179 below the model above
Placed by gap: in dollars, across only the models shown.
- 1Gemini 4 Carbon#1 of 72 on Vending-Bench 2Gemini 4 Carbon: Vending-Bench 2 $16,484, 1 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gemini-4-carbon_unknown$16,484
- 2GPT-6 Astra#2 of 72 on Vending-Bench 2GPT-6 Astra: Vending-Bench 2 $15,515, 2 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-6-astra_unknown$15,515
- 3GPT-6 Sol#3 of 72 on Vending-Bench 2GPT-6 Sol: Vending-Bench 2 $14,428, 3 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-6-sol_unknown$14,428
- 4Gemini 4 Argon#4 of 72 on Vending-Bench 2Gemini 4 Argon: Vending-Bench 2 $13,718, 4 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gemini-4-argon_unknown$13,718
- 5GPT-6.1 Sol#5 of 72 on Vending-Bench 2GPT-6.1 Sol: Vending-Bench 2 $12,861, 5 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-6.1-sol_unknown$12,861
- 6Claude Opus 5#6 of 72 on Vending-Bench 2Claude Opus 5: Vending-Bench 2 $11,182, 6 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as claude-opus-5_unknown$11,182
- 7Claude Sonnet 5.5#7 of 72 on Vending-Bench 2Claude Sonnet 5.5: Vending-Bench 2 $11,165, 7 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as claude-sonnet-5-5_unknown$11,165
- 8Grok 4.7#9 of 72 on Vending-Bench 2Grok 4.7: Vending-Bench 2 $10,537, 9 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as grok-4.7_unknown$10,537
- 9Muse Wonder#11 of 72 on Vending-Bench 2Muse Wonder: Vending-Bench 2 $9,432, 11 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as muse-wonder-1.0_unknown$9,432
- 10Claude Opus 5.5#12 of 72 on Vending-Bench 2Claude Opus 5.5: Vending-Bench 2 $9,235, 12 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as claude-opus-5-5_unknown$9,235
- 11GPT-6.1 Terra#13 of 72 on Vending-Bench 2GPT-6.1 Terra: Vending-Bench 2 $9,107, 13 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-6.1-terra_unknown$9,107
- 12Grok 4.6#14 of 72 on Vending-Bench 2Grok 4.6: Vending-Bench 2 $9,047, 14 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as grok-4.6_unknown$9,047
- 13GLM-5.3#16 of 72 on Vending-Bench 2GLM-5.3: Vending-Bench 2 $8,164, 16 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as glm-5.3_unknown$8,164
- 14GPT-5.6 Terra#19 of 72 on Vending-Bench 2GPT-5.6 Terra: Vending-Bench 2 $7,343, 19 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-5.6-terra_unknown$7,343
- 15Claude Fable 5.1#30 of 72 on Vending-Bench 2Claude Fable 5.1: Vending-Bench 2 $5,422, 30 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as claude-fable-5-1_unknown$5,422
- 16Kimi K3#32 of 72 on Vending-Bench 2Kimi K3: Vending-Bench 2 $5,165, 32 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as kimi-k3_unknown$5,165
- 17Gemini 3.8 Flash#34 of 72 on Vending-Bench 2Gemini 3.8 Flash: Vending-Bench 2 $5,094, 34 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gemini-3.8-flash_unknown$5,094
- 18GPT-5.6 Luna#44 of 72 on Vending-Bench 2GPT-5.6 Luna: Vending-Bench 2 $4,095, 44 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-5.6-luna_unknown$4,095
- 19Claude Haiku 5.5#49 of 72 on Vending-Bench 2Claude Haiku 5.5: Vending-Bench 2 $3,602, 49 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as claude-haiku-5-5_unknown$3,602
- 20MiniMax-M3#54 of 72 on Vending-Bench 2MiniMax-M3: Vending-Bench 2 $2,158, 54 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as MiniMax-M3$2,158
- 21gpt-oss-120b#70 of 72 on Vending-Bench 2gpt-oss-120b: Vending-Bench 2 −$22, 70 of 72 on Vending-Bench 2, listed by Vending-Bench 2 as gpt-oss-120b−$22
51 more Vending-Bench 2 entries, under Vending-Bench 2’s own names
- #8 claude-opus-4-7_unknown $10,937
- #10 gpt-5.6-sol_unknown $9,619
- #15 glm-5.2_unknown $8,314
- #17 claude-opus-4-6_unknown $8,018
- #18 gpt-5.5_unknown $7,524
- #20 claude-sonnet-4-6_unknown $7,204
- #21 muse-spark-1.1 $6,520
- #22 claude-sonnet-5_unknown $6,378
- #23 kimi-k2.6 $6,205
- #24 gpt-5.4-2026-03-05_unknown $6,144
- #25 gpt-5.3-codex $5,940
- #26 claude-opus-4-8_unknown $5,787
- #27 claude-fable-5_high $5,680
- #28 glm-5.1 $5,634
- #29 gemini-3-pro-preview $5,478
- #31 gemini-3.5-flash_unknown $5,396
- #33 qwen3.6-plus $5,115
- #35 kimi-k2.7-code $5,083
- #36 claude-fable-5_low $5,019
- #37 claude-opus-4-5-20251101_unknown $4,967
- #38 claude-fable-5_max $4,967
- #39 grok-4-20 $4,663
- #40 claude-fable-5 $4,530
- #41 glm-5 $4,432
- #42 claude-fable-5_medium $4,340
- #43 qwen3.6-max-preview $4,254
- #45 grok-4.5_unknown $3,887
- #46 claude-sonnet-4-5-20250929_unknown $3,839
- #47 gemini-3.1-pro-preview-customtools $3,774
- #48 gemini-3-flash-preview $3,635
- #50 gpt-5.2-2025-12-11_unknown $3,591
- #51 deepseek-v4-pro_unknown $3,285
- #52 claude-opus-4-8_max $2,992
- #53 glm-4.7 $2,377
- #55 gpt-5.1-2025-11-13_unknown $1,473
- #56 kimi-k2.5 $1,198
- #57 grok-4-1-fast-reasoning $1,107
- #58 DeepSeek-V3.2-Exp_unknown $1,034
- #59 gemini-3.1-pro-preview $911
- #60 gemini-2.5-pro $574
- #61 gemini-2.5-flash $549
- #62 qwen3.5-flash $463
- #63 claude-haiku-4-5-20251001_unknown $459
- #64 qwen3.5-27B $202
- #65 MiniMax-M2 $161
- #66 qwen3-max-2025-09-23 $72
- #67 grok-4.3_unknown $35
- #68 qwen3.5-plus $1
- #69 Qwen3-235B-A22B-Thinking-2507 −$11
- #71 MiniMax-M2.5 −$23
- #72 gpt-5-mini-2025-08-07_unknown −$31
The score is the bank balance in dollars after the simulated year, averaged over runs, as Epoch AI publishes Andon Labs’ results. A model that loses money ends below its $500 start.
21 of Vending-Bench 2’s 72 entries are models this site shows, ranked above under the site’s names. The other 51 are listed under Vending-Bench 2’s own names.
Data: Vending-Bench 2 by Andon Labs, results from Epoch AI’s open benchmark data, under CC BY 4.0, data of . Part of External Composite and Everything Composite. About this source
ADEMark measures the app around the model, not the model. Every app here runs the same model, Claude Opus 5.5, effort high, on one Mac mini M4, 16 GB, macOS 27, and is timed on how fast it starts, how far typing lags while five agents stream, how much memory it holds and how many tokens it sends for a plain “hi”. Artificial Analysis and the Gauntlet measure models; neither measures the harness a model works in, which is what decides how an agent feels to use. Because it ranks apps, it is in neither External Composite nor Everything Composite.
ADEMark · agentic development apps, one model · data of · CC BY 4.0
| App | Startup | Typing lag, 5 agents | Memory, 5 idle threads | Tokens for “hi” |
|---|---|---|---|---|
| Bare CLIcontrol, never ranked | 0.69 s | 79 ms | 1,013 MB | 38.2K |
| bb0.49.0 | 2.25 s | 62 ms | 1,527 MB | 24.5K |
| Capy0.8.0 | 2.00 s | Not measured | 593 MB | Not measured |
| Claude desktop4.23118.0 | 2.43 s | Not measured | 1,696 MB | 75.5K |
| Cloudroom129.0.0 for tokens, 150.0.0 for C1 | Not measured | Not measured | Not measured | 41.5K |
| Diri0.11.2 | 2.26 s | 72 ms | Not measured | 39.5K |
| Emdash1.2.10 | 4.25 s | 83 ms | 2,334 MB | 77.0K |
| MonoCode0.16.0 | 1.80 s | 62 ms | 419 MB | 44.6K |
| Orca1.4.249 | Not measured | 69 ms | 1,486 MB | 37.7K |
| Paseo0.15.0 | 4.22 s | 65 ms | 1,262 MB | 22.2K First |
| super.engineeringbuild 15aae8f8 | 1.76 s First | 72 ms | 134 MB First | 29.6K |
| Superset2.0.8, now named Paradev | Did not finish | Did not finish | Did not finish | Did not finish |
| Synara1.1.0 | 7.13 s | 51 ms | 1,348 MB | 30.6K |
| T3 Code0.1.8 beta | 2.74 s | 48 ms First | 1,356 MB | 52.7K |
| Traycer1.5.3 | 11.35 s | 50 ms | 1,874 MB | 68.4K |
| Zeron0.2.136 | 1.92 s | 59 ms | 951 MB | 40.5K |
Each figure is the median of an app’s valid runs. Lower is better on all four, and each test is ranked on its own, so there is no overall winner: First marks the best app on each test. The Bare CLI, Claude Code alone in a terminal, is ADEMark’s control and is never ranked. Did not finish and Not measured are ADEMark’s own words for a gap; its site gives the cause of each.
Data: ADEMark’s published results, run control-2026-11-10T08-31-19Z, under CC BY 4.0, data of . In neither External Composite nor Everything Composite. About this source
External Composite puts the outside benchmarks together, and leaves out Artificial Analysis and the Gauntlet. Each model’s place in each source is its percentile among every model that source ranks, 100 for the source’s top model and 0 for its bottom, so an arena rating, a share of math problems solved and a dollar balance count alike. The dot is the mean of those percentiles, and the open marks are the parts. Only the 29 models in at least two of LMArena, FrontierMath and Vending-Bench 2 are ranked; ADEMark ranks apps, not models, so it is not part of it.
Made from LMArena, FrontierMath and Vending-Bench 2 · newest data of · each under CC BY 4.0
Gemini 4 Carbon sits 3.4× closer to 100 than Gemini 4 Argon.
- 1Gemini 4 CarbonGemini 4 Carbon: External Composite 98, the leader, by a clear lead
- 2Gemini 4 ArgonGemini 4 Argon: External Composite 94, 3.4 times the distance to 100 of the model above
- 3Claude Opus 5.5Claude Opus 5.5: External Composite 93, 1.1 times the distance to 100 of the model above
- 4GPT-6 AstraGPT-6 Astra: External Composite 93, 1.1 times the distance to 100 of the model above
- 5GPT-6.1 SolGPT-6.1 Sol: External Composite 92, 1.1 times the distance to 100 of the model above
- 6Claude Opus 5Claude Opus 5: External Composite 92, 1.0 times the distance to 100 of the model above
- 7Claude Sonnet 5.5Claude Sonnet 5.5: External Composite 91, 1.1 times the distance to 100 of the model above
- 8GPT-6.1 TerraGPT-6.1 Terra: External Composite 89, 1.3 times the distance to 100 of the model above
- 9Muse WonderMuse Wonder: External Composite 88, 1.1 times the distance to 100 of the model above
- 10Muse Spark 1.3Muse Spark 1.3: External Composite 85, 1.3 times the distance to 100 of the model above
- 11GPT-6 SolGPT-6 Sol: External Composite 84, 1.1 times the distance to 100 of the model above
- 12Claude Fable 5.1Claude Fable 5.1: External Composite 83, 1.1 times the distance to 100 of the model above
- 13Kimi K3.1 ProKimi K3.1 Pro: External Composite 83, 1.0 times the distance to 100 of the model above
- 14GPT-5.6 TerraGPT-5.6 Terra: External Composite 82, 1.1 times the distance to 100 of the model above
- 15GLM-5.3GLM-5.3: External Composite 79, 1.2 times the distance to 100 of the model above
- 16Grok 4.6Grok 4.6: External Composite 72, 1.3 times the distance to 100 of the model above
- 17Kimi K3Kimi K3: External Composite 72, 1.0 times the distance to 100 of the model above
- 18Gemini 3.8 FlashGemini 3.8 Flash: External Composite 71, 1.1 times the distance to 100 of the model above
- 19GPT-6 LunaGPT-6 Luna: External Composite 69, 1.0 times the distance to 100 of the model above
- 20GLM-5.3-FlashGLM-5.3-Flash: External Composite 69, 1.0 times the distance to 100 of the model above
- 21GPT-5.6 LunaGPT-5.6 Luna: External Composite 66, 1.1 times the distance to 100 of the model above
- 22Grok 4.7Grok 4.7: External Composite 64, 1.1 times the distance to 100 of the model above
- 23Mistral Large 4Mistral Large 4: External Composite 59, 1.1 times the distance to 100 of the model above
- 24Claude Haiku 5.5Claude Haiku 5.5: External Composite 57, 1.0 times the distance to 100 of the model above
- 25InklingInkling: External Composite 53, 1.1 times the distance to 100 of the model above
- 26Inkling SmallInkling Small: External Composite 53, 1.0 times the distance to 100 of the model above
- 27MiniMax-M3MiniMax-M3: External Composite 52, 1.0 times the distance to 100 of the model above
- 28Gemini 3.5 Flash-LiteGemini 3.5 Flash-Lite: External Composite 49, 1.1 times the distance to 100 of the model above
- 29gpt-oss-120bgpt-oss-120b: External Composite 29, 1.4 times the distance to 100 of the model above
Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.
External CompositeLMArena percentileFrontierMath percentileVending-Bench 2 percentile
- 1Gemini 4 Carbon◇ LMArena 100□ FrontierMath 95○ Vending-Bench 2 100Gemini 4 Carbon: External Composite 98, the mean of its percentiles in LMArena 100, FrontierMath 95 and Vending-Bench 2 100 (LMArena 1538, FrontierMath 90.5% and Vending-Bench 2 $16,484)
- 2Gemini 4 Argon◇ LMArena 100□ FrontierMath 86○ Vending-Bench 2 95Gemini 4 Argon: External Composite 94, the mean of its percentiles in LMArena 100, FrontierMath 86 and Vending-Bench 2 95 (LMArena 1529, FrontierMath 86.0% and Vending-Bench 2 $13,718)
- 3Claude Opus 5.5◇ LMArena 99□ FrontierMath 97○ Vending-Bench 2 83Claude Opus 5.5: External Composite 93, the mean of its percentiles in LMArena 99, FrontierMath 97 and Vending-Bench 2 83 (LMArena 1514, FrontierMath 91.2% and Vending-Bench 2 $9,235)
- 4GPT-6 Astra◇ LMArena 82□ FrontierMath 98○ Vending-Bench 2 98GPT-6 Astra: External Composite 93, the mean of its percentiles in LMArena 82, FrontierMath 98 and Vending-Bench 2 98 (LMArena 1440, FrontierMath 93.7% and Vending-Bench 2 $15,515)
- 5GPT-6.1 Sol◇ LMArena 85□ FrontierMath 98○ Vending-Bench 2 94GPT-6.1 Sol: External Composite 92, the mean of its percentiles in LMArena 85, FrontierMath 98 and Vending-Bench 2 94 (LMArena 1445, FrontierMath 93.7% and Vending-Bench 2 $12,861)
- 6Claude Opus 5◇ LMArena 99□ FrontierMath 85○ Vending-Bench 2 92Claude Opus 5: External Composite 92, the mean of its percentiles in LMArena 99, FrontierMath 85 and Vending-Bench 2 92 (LMArena 1507, FrontierMath 85.6% and Vending-Bench 2 $11,182)
- 7Claude Sonnet 5.5◇ LMArena 92□ FrontierMath 90○ Vending-Bench 2 91Claude Sonnet 5.5: External Composite 91, the mean of its percentiles in LMArena 92, FrontierMath 90 and Vending-Bench 2 91 (LMArena 1469, FrontierMath 88.8% and Vending-Bench 2 $11,165)
- 8GPT-6.1 Terra□ FrontierMath 96○ Vending-Bench 2 82GPT-6.1 Terra: External Composite 89, the mean of its percentiles in FrontierMath 96 and Vending-Bench 2 82 (FrontierMath 90.9% and Vending-Bench 2 $9,107)
- 9Muse Wonder◇ LMArena 99□ FrontierMath 81○ Vending-Bench 2 85Muse Wonder: External Composite 88, the mean of its percentiles in LMArena 99, FrontierMath 81 and Vending-Bench 2 85 (LMArena 1503, FrontierMath 82.1% and Vending-Bench 2 $9,432)
- 10Muse Spark 1.3◇ LMArena 97□ FrontierMath 73Muse Spark 1.3: External Composite 85, the mean of its percentiles in LMArena 97 and FrontierMath 73 (LMArena 1492 and FrontierMath 74.4%)
- 11GPT-6 Sol◇ LMArena 62□ FrontierMath 92○ Vending-Bench 2 97GPT-6 Sol: External Composite 84, the mean of its percentiles in LMArena 62, FrontierMath 92 and Vending-Bench 2 97 (LMArena 1397, FrontierMath 89.8% and Vending-Bench 2 $14,428)
- 12Claude Fable 5.1◇ LMArena 99□ FrontierMath 93○ Vending-Bench 2 56Claude Fable 5.1: External Composite 83, the mean of its percentiles in LMArena 99, FrontierMath 93 and Vending-Bench 2 56 (LMArena 1512, FrontierMath 90.2% and Vending-Bench 2 $5,422)
- 13Kimi K3.1 Pro◇ LMArena 90□ FrontierMath 76Kimi K3.1 Pro: External Composite 83, the mean of its percentiles in LMArena 90 and FrontierMath 76 (LMArena 1459 and FrontierMath 78.2%)
- 14GPT-5.6 Terra◇ LMArena 87□ FrontierMath 86○ Vending-Bench 2 73GPT-5.6 Terra: External Composite 82, the mean of its percentiles in LMArena 87, FrontierMath 86 and Vending-Bench 2 73 (LMArena 1449, FrontierMath 86.0% and Vending-Bench 2 $7,343)
- 15GLM-5.3◇ LMArena 94□ FrontierMath 65○ Vending-Bench 2 77GLM-5.3: External Composite 79, the mean of its percentiles in LMArena 94, FrontierMath 65 and Vending-Bench 2 77 (LMArena 1475, FrontierMath 68.8% and Vending-Bench 2 $8,164)
- 16Grok 4.6◇ LMArena 77□ FrontierMath 60○ Vending-Bench 2 80Grok 4.6: External Composite 72, the mean of its percentiles in LMArena 77, FrontierMath 60 and Vending-Bench 2 80 (LMArena 1430, FrontierMath 66.0% and Vending-Bench 2 $9,047)
- 17Kimi K3◇ LMArena 94□ FrontierMath 70○ Vending-Bench 2 53Kimi K3: External Composite 72, the mean of its percentiles in LMArena 94, FrontierMath 70 and Vending-Bench 2 53 (LMArena 1475, FrontierMath 72.2% and Vending-Bench 2 $5,165)
- 18Gemini 3.8 Flash◇ LMArena 98□ FrontierMath 64○ Vending-Bench 2 50Gemini 3.8 Flash: External Composite 71, the mean of its percentiles in LMArena 98, FrontierMath 64 and Vending-Bench 2 50 (LMArena 1500, FrontierMath 68.4% and Vending-Bench 2 $5,094)
- 19GPT-6 Luna◇ LMArena 60□ FrontierMath 78GPT-6 Luna: External Composite 69, the mean of its percentiles in LMArena 60 and FrontierMath 78 (LMArena 1388 and FrontierMath 78.9%)
- 20GLM-5.3-Flash◇ LMArena 93□ FrontierMath 45GLM-5.3-Flash: External Composite 69, the mean of its percentiles in LMArena 93 and FrontierMath 45 (LMArena 1472 and FrontierMath 55.8%)
- 21GPT-5.6 Luna◇ LMArena 78□ FrontierMath 81○ Vending-Bench 2 41GPT-5.6 Luna: External Composite 66, the mean of its percentiles in LMArena 78, FrontierMath 81 and Vending-Bench 2 41 (LMArena 1431, FrontierMath 82.1% and Vending-Bench 2 $4,095)
- 22Grok 4.7◇ LMArena 63□ FrontierMath 41○ Vending-Bench 2 88Grok 4.7: External Composite 64, the mean of its percentiles in LMArena 63, FrontierMath 41 and Vending-Bench 2 88 (LMArena 1400, FrontierMath 53.0% and Vending-Bench 2 $10,537)
- 23Mistral Large 4◇ LMArena 68□ FrontierMath 49Mistral Large 4: External Composite 59, the mean of its percentiles in LMArena 68 and FrontierMath 49 (LMArena 1411 and FrontierMath 58.6%)
- 24Claude Haiku 5.5◇ LMArena 63□ FrontierMath 75○ Vending-Bench 2 33Claude Haiku 5.5: External Composite 57, the mean of its percentiles in LMArena 63, FrontierMath 75 and Vending-Bench 2 33 (LMArena 1398, FrontierMath 75.1% and Vending-Bench 2 $3,602)
- 25Inkling◇ LMArena 83□ FrontierMath 24Inkling: External Composite 53, the mean of its percentiles in LMArena 83 and FrontierMath 24 (LMArena 1443 and FrontierMath 33.3%)
- 26Inkling Small◇ LMArena 69□ FrontierMath 36Inkling Small: External Composite 53, the mean of its percentiles in LMArena 69 and FrontierMath 36 (LMArena 1414 and FrontierMath 46.3%)
- 27MiniMax-M3◇ LMArena 76○ Vending-Bench 2 27MiniMax-M3: External Composite 52, the mean of its percentiles in LMArena 76 and Vending-Bench 2 27 (LMArena 1428 and Vending-Bench 2 $2,158)
- 28Gemini 3.5 Flash-Lite◇ LMArena 79□ FrontierMath 19Gemini 3.5 Flash-Lite: External Composite 49, the mean of its percentiles in LMArena 79 and FrontierMath 19 (LMArena 1435 and FrontierMath 26.0%)
- 29gpt-oss-120b◇ LMArena 55○ Vending-Bench 2 3gpt-oss-120b: External Composite 29, the mean of its percentiles in LMArena 55 and Vending-Bench 2 3 (LMArena 1366 and Vending-Bench 2 −$22)
Not ranked, with only one outside source: Claude Fable 5.2 (FrontierMath), Command A+ (LMArena), DeepSeek V4.1 Flash 1017 (FrontierMath), DeepSeek V4 Pro 0813 (FrontierMath), Gemini 3.1 Flash-Lite (FrontierMath), Gemini 4 Hydrogen (LMArena), Gemma 4 31B (LMArena), Granite 4.2 3B (LMArena), Granite 4.2 8B (LMArena), Inkling 2 Small (LMArena), MiMo-V2.6-Flash (LMArena), MiMo-V2.6-Pro (LMArena), Mistral Medium 3.5 (LMArena), Muse Glimmer (LMArena), Muse Spark Open (FrontierMath), Qwen3.8 27B (LMArena), Qwen3.8 Max (0902) (FrontierMath), Solar Pro 4 (LMArena) and Step 5 Preview (LMArena).
Each source’s percentiles are taken among every model it ranks: 365 in LMArena, 92 in FrontierMath and 67 in Vending-Bench 2, a model with several settings counted once at its best. Nothing here is measured anew.
Data: LMArena’s text leaderboard, from its open dataset, under CC BY 4.0; FrontierMath by Epoch AI, from Epoch AI’s open benchmark data, under CC BY 4.0; Vending-Bench 2 by Andon Labs, results from Epoch AI’s open benchmark data, under CC BY 4.0. About External Composite
Everything Composite puts the outside benchmarks and our own together. Each model’s dot is the mean of two percentiles: where its External Composite stands among every External Composite model, and where its Gauntlet Composite stands among every Gauntlet Composite model. Both parts are marked and named on its row. Only the 7 models with both are ranked.
Made from External Composite (LMArena, FrontierMath and Vending-Bench 2) and Gauntlet Composite (Gauntlet Science, Gauntlet Ship and Gauntlet Teach)
Claude Opus 5.5 sits 2.5× closer to 100 than Claude Fable 5.1.
- 1Claude Opus 5.5Claude Opus 5.5: Everything Composite 85, the leader, by a clear lead
- 2Claude Fable 5.1Claude Fable 5.1: Everything Composite 64, 2.5 times the distance to 100 of the model above
- 3Claude Sonnet 5.5Claude Sonnet 5.5: Everything Composite 62, 1.1 times the distance to 100 of the model above
- 4GPT-6.1 SolGPT-6.1 Sol: Everything Composite 60, 1.1 times the distance to 100 of the model above
- 5GPT-6 AstraGPT-6 Astra: Everything Composite 56, 1.1 times the distance to 100 of the model above
- 6GPT-6.1 TerraGPT-6.1 Terra: Everything Composite 38, 1.4 times the distance to 100 of the model above
- 7Claude Haiku 5.5Claude Haiku 5.5: Everything Composite 14, 1.4 times the distance to 100 of the model above
Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.
Everything CompositeExternal Composite percentileGauntlet Composite percentile
- 1Claude Opus 5.5◇ External 93□ Gauntlet 78Claude Opus 5.5: Everything Composite 85, the mean of its External Composite percentile 93 (External Composite 93) and its Gauntlet Composite percentile 78 (Gauntlet Composite 76)
- 2Claude Fable 5.1◇ External 61□ Gauntlet 67Claude Fable 5.1: Everything Composite 64, the mean of its External Composite percentile 61 (External Composite 83) and its Gauntlet Composite percentile 67 (Gauntlet Composite 74)
- 3Claude Sonnet 5.5◇ External 79□ Gauntlet 44Claude Sonnet 5.5: Everything Composite 62, the mean of its External Composite percentile 79 (External Composite 91) and its Gauntlet Composite percentile 44 (Gauntlet Composite 72)
- 4GPT-6.1 Sol◇ External 86□ Gauntlet 33GPT-6.1 Sol: Everything Composite 60, the mean of its External Composite percentile 86 (External Composite 92) and its Gauntlet Composite percentile 33 (Gauntlet Composite 71)
- 5GPT-6 Astra◇ External 89□ Gauntlet 22GPT-6 Astra: Everything Composite 56, the mean of its External Composite percentile 89 (External Composite 93) and its Gauntlet Composite percentile 22 (Gauntlet Composite 67)
- 6GPT-6.1 Terra◇ External 75□ Gauntlet 0GPT-6.1 Terra: Everything Composite 38, the mean of its External Composite percentile 75 (External Composite 89) and its Gauntlet Composite percentile 0 (Gauntlet Composite 44)
- 7Claude Haiku 5.5◇ External 18□ Gauntlet 11Claude Haiku 5.5: Everything Composite 14, the mean of its External Composite percentile 18 (External Composite 57) and its Gauntlet Composite percentile 11 (Gauntlet Composite 56)
Not ranked, with no Gauntlet Composite score: Claude Opus 5, Gemini 3.5 Flash-Lite, Gemini 3.8 Flash, Gemini 4 Argon, Gemini 4 Carbon, GLM-5.3, GLM-5.3-Flash, GPT-5.6 Luna, GPT-5.6 Terra, GPT-6 Luna, GPT-6 Sol, gpt-oss-120b, Grok 4.6, Grok 4.7, Inkling, Inkling Small, Kimi K3, Kimi K3.1 Pro, MiniMax-M3, Mistral Large 4, Muse Spark 1.3 and Muse Wonder. Not ranked, with no External Composite score: Claude Fable 5.2, GPT-6.2 Astra and GPT-6.2 Sol.
Percentiles put two very different rankings on one footing: External Composite ranks the models the outside sources share, and Gauntlet Composite the models our own benchmarks ran. Nothing here is measured anew.
Data: LMArena’s text leaderboard, from its open dataset, under CC BY 4.0; FrontierMath by Epoch AI, from Epoch AI’s open benchmark data, under CC BY 4.0; Vending-Bench 2 by Andon Labs, results from Epoch AI’s open benchmark data, under CC BY 4.0; and Unsupervised Learning’s Gauntlet Science, Gauntlet Ship and Gauntlet Teach. About Everything Composite
No models selected.
Artificial Analysis lists no cost for these models, so there is nothing to place by cost.
Each model is one dot: the plain mean, on one 0–100 scale, of every part switched on above that it has: its Artificial Analysis Intelligence Index averaged over its efforts, and its Gauntlet Science, Gauntlet Ship and Gauntlet Teach scores, so each counts equally. A model with no Gauntlet score switched on is left out. Cost is Artificial Analysis’s cost per index task, averaged over the same efforts. It shows the models picked in the Models list above. Hover a mark for every part.