ai benchmarks

Real-world AI performance metrics

Gauntlet Composite updated

Gauntlet Science updated

Gauntlet Ship updated

Gauntlet Teach updated

Gauntlet Judgment updated

Artificial Analysis data updated

Each line is one model, climbing from its lowest to its highest reasoning effort: up scores higher, right costs more per task.

  1. Pick models in the Models list.
  2. Hover or tap a dot for its effort, score and cost.
  3. Cost runs on a log scale: each equal step right is the same multiple of price.

Every model ranked by the best score it reaches within your budget, with what that setting costs per task.

  1. Drag the budget to the most you’ll pay per task.
  2. Dots run low → max effort; rings are over budget.
  3. Tap the dot by a name to add it to Curves.

The cheapest way to reach each score. Each white step is a setting nothing cheaper matches; grey dots are beaten for less.

  1. Find your score on the left; read across to the line.
  2. The table shows what each next step up costs.
  3. Pick models to see what beats their settings.

How fast reaching a score is getting cheaper: each line is the lowest cost per task to reach it with what was out by each date.

  1. Lower is cheaper; each drop is a cheaper release.
  2. The right-hand label is today’s cheapest price.
  3. Intelligence, Coding or Agentic changes the score.

Every change Artificial Analysis made to the data, newest first.

  1. Click a model to see its past scores.

Gauntlet Science measures whether a model can do science on something it has never seen. Every run generates 24 worlds fresh, each with its own hidden law, so there is nothing to memorise. In each world the model reads noisy observations, runs 6 experiments it designs itself, states the law it found, then makes 20 predictions with intervals, half of them beyond anything it measured.

GPT-6.2 Astra sits 1.0× closer to 100 than Claude Fable 5.2, a paired gap of −0.5 points (95% −3.6 to +2.6).

  1. Ceiling scriptreferenceCeiling script, a reference script checked on every item: Gauntlet Science score 93
  2. 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Science score 90, the leader
  3. 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Science score 89, 1.0 times the distance to 100 of the model above, −0.5 points from the leader, 95% interval −3.6 to +2.6, not separated
  4. 3GPT-6.2 SolGPT-6.2 Sol: Gauntlet Science score 86, 1.3 times the distance to 100 of the model above, −3.8 points from the leader, 95% interval −5.6 to −1.9, separated
  5. 4GPT-6 AstraGPT-6 Astra: Gauntlet Science score 85, 1.0 times the distance to 100 of the model above, −4.3 points from the leader, 95% interval −7.0 to −1.7, separated
  6. 5Claude Opus 5.5Claude Opus 5.5: Gauntlet Science score 82, 1.2 times the distance to 100 of the model above, −7.1 points from the leader, 95% interval −11.5 to −2.7, separated
  7. 6Claude Fable 5.1Claude Fable 5.1: Gauntlet Science score 82, 1.0 times the distance to 100 of the model above, −7.4 points from the leader, 95% interval −12.3 to −2.5, separated
  8. 7Gemini 4 NeonGemini 4 Neon: Gauntlet Science score 81, 1.1 times the distance to 100 of the model above, −8.6 points from the leader, 95% interval −21.5 to +4.2, not separated
  9. 8GPT-6.1 SolGPT-6.1 Sol: Gauntlet Science score 81, 1.0 times the distance to 100 of the model above, −8.8 points from the leader, 95% interval −14.6 to −3.1, separated
  10. 9Gemini 4 CarbonGemini 4 Carbon: Gauntlet Science score 78, 1.1 times the distance to 100 of the model above, −11.5 points from the leader, 95% interval −24.4 to +1.5, not separated
  11. 10Muse WonderMuse Wonder: Gauntlet Science score 78, 1.0 times the distance to 100 of the model above, −11.7 points from the leader, 95% interval −17.7 to −5.6, separated
  12. 11Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Science score 73, 1.2 times the distance to 100 of the model above, −16.4 points from the leader, 95% interval −22.2 to −10.6, separated
  13. 12Gemini 4 ArgonGemini 4 Argon: Gauntlet Science score 66, 1.3 times the distance to 100 of the model above, −23.6 points from the leader, 95% interval −34.4 to −12.8, separated
  14. 13Grok 4.7Grok 4.7: Gauntlet Science score 64, 1.1 times the distance to 100 of the model above, −25.5 points from the leader, 95% interval −36.0 to −15.0, separated
  15. 14Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Science score 60, 1.1 times the distance to 100 of the model above, −29.6 points from the leader, 95% interval −38.5 to −20.7, separated
  16. 15GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Science score 59, 1.0 times the distance to 100 of the model above, −30.9 points from the leader, 95% interval −41.9 to −19.8, separated
  17. 16GPT-6 LunaGPT-6 Luna: Gauntlet Science score 44, 1.4 times the distance to 100 of the model above, −45.5 points from the leader, 95% interval −58.7 to −32.2, separated
  18. Floor scriptreferenceFloor script, a reference script checked on every item: Gauntlet Science score 37

Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.

  1. Ceiling scriptreferenceCeiling script, a reference script: score 93, 95% interval 90 to 9795% 90–97
  2. 1GPT-6.2 AstraGPT-6.2 Astra: score 90, 95% interval 83 to 96, the leader95% 83–96Leader
  3. 2Claude Fable 5.2Claude Fable 5.2: score 89, 95% interval 84 to 94, −0.5 points from the leader, 95% interval −3.6 to +2.6, not separated95% 84–94−0.595% −3.6 to +2.6Not separated
  4. 3GPT-6.2 SolGPT-6.2 Sol: score 86, 95% interval 79 to 92, −3.8 points from the leader, 95% interval −5.6 to −1.9, separated95% 79–92−3.895% −5.6 to −1.9● Separated
  5. 4GPT-6 AstraGPT-6 Astra: score 85, 95% interval 79 to 92, −4.3 points from the leader, 95% interval −7.0 to −1.7, separated95% 79–92−4.395% −7.0 to −1.7● Separated
  6. 5Claude Opus 5.5Claude Opus 5.5: score 82, 95% interval 74 to 91, −7.1 points from the leader, 95% interval −11.5 to −2.7, separated95% 74–91−7.195% −11.5 to −2.7● Separated
  7. 6Claude Fable 5.1Claude Fable 5.1: score 82, 95% interval 74 to 91, −7.4 points from the leader, 95% interval −12.3 to −2.5, separated95% 74–91−7.495% −12.3 to −2.5● Separated
  8. 7Gemini 4 NeonGemini 4 Neon: score 81, 95% interval 69 to 93, −8.6 points from the leader, 95% interval −21.5 to +4.2, not separated95% 69–93−8.695% −21.5 to +4.2Not separated
  9. 8GPT-6.1 SolGPT-6.1 Sol: score 81, 95% interval 73 to 88, −8.8 points from the leader, 95% interval −14.6 to −3.1, separated95% 73–88−8.895% −14.6 to −3.1● Separated
  10. 9Gemini 4 CarbonGemini 4 Carbon: score 78, 95% interval 65 to 91, −11.5 points from the leader, 95% interval −24.4 to +1.5, not separated95% 65–91−11.595% −24.4 to +1.5Not separated
  11. 10Muse WonderMuse Wonder: score 78, 95% interval 69 to 87, −11.7 points from the leader, 95% interval −17.7 to −5.6, separated95% 69–87−11.795% −17.7 to −5.6● Separated
  12. 11Claude Sonnet 5.5Claude Sonnet 5.5: score 73, 95% interval 65 to 81, −16.4 points from the leader, 95% interval −22.2 to −10.6, separated95% 65–81−16.495% −22.2 to −10.6● Separated
  13. 12Gemini 4 ArgonGemini 4 Argon: score 66, 95% interval 55 to 77, −23.6 points from the leader, 95% interval −34.4 to −12.8, separated95% 55–77−23.695% −34.4 to −12.8● Separated
  14. 13Grok 4.7Grok 4.7: score 64, 95% interval 51 to 77, −25.5 points from the leader, 95% interval −36.0 to −15.0, separated95% 51–77−25.595% −36.0 to −15.0● Separated
  15. 14Claude Haiku 5.5Claude Haiku 5.5: score 60, 95% interval 50 to 69, −29.6 points from the leader, 95% interval −38.5 to −20.7, separated95% 50–69−29.695% −38.5 to −20.7● Separated
  16. 15GPT-6.1 TerraGPT-6.1 Terra: score 59, 95% interval 48 to 69, −30.9 points from the leader, 95% interval −41.9 to −19.8, separated95% 48–69−30.995% −41.9 to −19.8● Separated
  17. 16GPT-6 LunaGPT-6 Luna: score 44, 95% interval 33 to 55, −45.5 points from the leader, 95% interval −58.7 to −32.2, separated95% 33–55−45.595% −58.7 to −32.2● Separated
  18. Floor scriptreferenceFloor script, a reference script: score 37, 95% interval 26 to 4795% 26–47

Code wrote each world and knows its law, so code scores every prediction against the hidden law: no judge reads an answer. A score of 100 is every prediction exact and certain; 0 is a miss or no answer. The worlds climb three rungs of difficulty (8, 8 and 8 worlds), and each score carries a 95% interval over the worlds. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are scripts, not models. The floor script runs random experiments and fits a straight line, so a model below it did no better than that. The ceiling script spreads its experiments to the edges and picks the best-fitting law from the families the worlds are drawn from, a practical top.

Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (separation, headroom, stability, length, cap, no cheap twin and effort). This run fails 1 of them (stability), so read its ranking as provisional.

Not ranked, because a model is ranked only when it finished every world: Kimi (2 of 24 cells never reached an answer: the provider failed).

Worlds that ended at a time or output limit are scored as unanswered, which lowers a row: Grok 4.7 on 3 of 24, Gemini 4 Neon on 2 of 24, Gemini 4 Carbon on 2 of 24 and Muse Wonder on 1 of 24.

Graded 2026-11-20; run f8b224a2, seed 20261112, protocol gauntlet-discovery-v3. Full methodology

Gauntlet Ship measures whether a model can finish real work. Each of the 55 items is a real task from Unsupervised Learning’s own working sessions, across 16 parts of a codebase the model has never seen. The model gets the working conversation as its ask and the repository as it stood before the fix, then has 30 steps of one plain tool loop to read, change and test the code before it says whether it is done.

Claude Fable 5.2 makes 1.1× fewer misses than Claude Fable 5.1, a paired gap of −3.6 points (95% −15.0 to +7.7).

  1. Accepted fixreferenceAccepted fix, a reference script checked on every item: Gauntlet Ship score 100
  2. 1Claude Fable 5.2Claude Fable 5.2: Gauntlet Ship score 71, hidden checks 83%, false done 19%, the leader
  3. 2Claude Fable 5.1Claude Fable 5.1: Gauntlet Ship score 67, hidden checks 77%, false done 30%, 1.1 times the misses of the model above, −3.6 points from the leader, 95% interval −15.0 to +7.7, not separated
  4. 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Ship score 66, hidden checks 74%, false done 26%, 1.1 times the misses of the model above, −5.5 points from the leader, 95% interval −17.3 to +6.4, not separated
  5. 4GPT-6.1 SolGPT-6.1 Sol: Gauntlet Ship score 64, hidden checks 76%, false done 30%, 1.1 times the misses of the model above, −7.3 points from the leader, 95% interval −19.6 to +5.0, not separated
  6. 5GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Ship score 64, hidden checks 76%, false done 31%, 1.0 times the misses of the model above, −7.3 points from the leader, 95% interval −18.5 to +3.9, not separated
  7. 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Ship score 60, hidden checks 72%, false done 39%, 1.1 times the misses of the model above, −10.9 points from the leader, 95% interval −21.9 to +0.1, not separated
  8. 7GPT-6.2 SolGPT-6.2 Sol: Gauntlet Ship score 55, hidden checks 68%, false done 40%, 1.1 times the misses of the model above, −16.4 points from the leader, 95% interval −26.2 to −6.5, separated
  9. 8Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Ship score 46, hidden checks 61%, false done 46%, 1.2 times the misses of the model above, −25.5 points from the leader, 95% interval −38.1 to −12.8, separated
  10. 9GPT-6 AstraGPT-6 Astra: Gauntlet Ship score 40, hidden checks 55%, false done 35%, 1.1 times the misses of the model above, −30.9 points from the leader, 95% interval −44.2 to −17.6, separated
  11. 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Ship score 40, hidden checks 52%, false done 40%, 1.0 times the misses of the model above, −30.9 points from the leader, 95% interval −45.2 to −16.6, separated
  12. No changereferenceNo change, a reference script checked on every item: Gauntlet Ship score 0

Placed by gap: equal distances are equal ratios of misses, across only the models shown.

  1. Accepted fixreferenceAccepted fix, a reference script: Gauntlet Ship score 100, 95% interval 100 to 10095% 100–100
  2. 1Claude Fable 5.2Hidden checks 83%False done 19%Claude Fable 5.2: Gauntlet Ship score 71, 95% interval 59 to 83, hidden checks 83%, false done 19%, the leader95% 59–83Leader
  3. 2Claude Fable 5.1Hidden checks 77%False done 30%Claude Fable 5.1: Gauntlet Ship score 67, 95% interval 55 to 80, hidden checks 77%, false done 30%, −3.6 points from the leader, 95% interval −15.0 to +7.7, not separated95% 55–80−3.695% −15.0 to +7.7Not separated
  4. 3Claude Opus 5.5Hidden checks 74%False done 26%Claude Opus 5.5: Gauntlet Ship score 66, 95% interval 53 to 78, hidden checks 74%, false done 26%, −5.5 points from the leader, 95% interval −17.3 to +6.4, not separated95% 53–78−5.595% −17.3 to +6.4Not separated
  5. 4GPT-6.1 SolHidden checks 76%False done 30%GPT-6.1 Sol: Gauntlet Ship score 64, 95% interval 51 to 77, hidden checks 76%, false done 30%, −7.3 points from the leader, 95% interval −19.6 to +5.0, not separated95% 51–77−7.395% −19.6 to +5.0Not separated
  6. 5GPT-6.2 AstraHidden checks 76%False done 31%GPT-6.2 Astra: Gauntlet Ship score 64, 95% interval 51 to 77, hidden checks 76%, false done 31%, −7.3 points from the leader, 95% interval −18.5 to +3.9, not separated95% 51–77−7.395% −18.5 to +3.9Not separated
  7. 6Claude Sonnet 5.5Hidden checks 72%False done 39%Claude Sonnet 5.5: Gauntlet Ship score 60, 95% interval 47 to 73, hidden checks 72%, false done 39%, −10.9 points from the leader, 95% interval −21.9 to +0.1, not separated95% 47–73−10.995% −21.9 to +0.1Not separated
  8. 7GPT-6.2 SolHidden checks 68%False done 40%GPT-6.2 Sol: Gauntlet Ship score 55, 95% interval 41 to 68, hidden checks 68%, false done 40%, −16.4 points from the leader, 95% interval −26.2 to −6.5, separated95% 41–68−16.495% −26.2 to −6.5● Separated
  9. 8Claude Haiku 5.5Hidden checks 61%False done 46%Claude Haiku 5.5: Gauntlet Ship score 46, 95% interval 32 to 59, hidden checks 61%, false done 46%, −25.5 points from the leader, 95% interval −38.1 to −12.8, separated95% 32–59−25.595% −38.1 to −12.8● Separated
  10. 9GPT-6 AstraHidden checks 55%False done 35%GPT-6 Astra: Gauntlet Ship score 40, 95% interval 27 to 53, hidden checks 55%, false done 35%, −30.9 points from the leader, 95% interval −44.2 to −17.6, separated95% 27–53−30.995% −44.2 to −17.6● Separated
  11. 10GPT-6.1 TerraHidden checks 52%False done 40%GPT-6.1 Terra: Gauntlet Ship score 40, 95% interval 27 to 53, hidden checks 52%, false done 40%, −30.9 points from the leader, 95% interval −45.2 to −16.6, separated95% 27–53−30.995% −45.2 to −16.6● Separated
  12. No changereferenceNo change, a reference script: Gauntlet Ship score 0, 95% interval 0 to 095% 0–0

Hidden checks, written from that conversation, decide each item. A check is kept only when it fails on the code before the accepted fix and passes after it, and checks are screened so they test nothing the conversation left open. An item is shipped when every hidden check passes and no test that passed before breaks. Code runs the checks and makes every number: no judge reads the work. The Gauntlet Ship score is the share of items shipped, with a 95% interval over the items. Hidden checks is the share of hidden checks a model passed, and false done is the share of its “done” claims that did not ship. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are scripts, not models. No change finishes at once and ships nothing, so a model at that row changed nothing that mattered. The accepted fix writes the fix that was actually accepted, and every item it does not ship is dropped, so each item counted here can be shipped.

Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (items valid, separation, headroom, stability, no cheap twin, length and effort). This run fails 1 of them (stability), so read its ranking as provisional.

Argon ran but is left out of this ranking: 47 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked. Carbon ran but is left out of this ranking: 54 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked. Neon ran but is left out of this ranking: 50 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked.

Grok 4.7 played but is not ranked (5 of 55 cells never reached an answer: the provider failed). Muse Wonder played but is not ranked (54 of 55 cells never reached an answer: the provider refused for rate or quota).

The items are private work, so this page shows counts and scores only: never an item’s conversation, its code, its checks or a model’s transcript.

Graded 2026-11-20; run f401c811, item set v4, protocol gauntlet-ship-v1. Full methodology

Gauntlet Composite puts Unsupervised Learning’s own benchmarks together. Each dot is one model’s plain mean of its three scores on 0–100: Gauntlet Science, Gauntlet Ship and Gauntlet Teach. The small marks on each row are those parts. Only the 10 models every one of these benchmarks ran are ranked, so every dot is made of the same parts. Every model runs at high effort; a model whose API has no effort setting runs at its own default.

GPT-6.2 Astra sits 1.1× closer to 100 than Claude Fable 5.2.

  1. 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Composite 81, the mean of Gauntlet Science 90, Gauntlet Ship 64 and Gauntlet Teach 91, the leader
  2. 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Composite 80, the mean of Gauntlet Science 89, Gauntlet Ship 71 and Gauntlet Teach 81, 1.1 times the distance to 100 of the model above
  3. 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Composite 76, the mean of Gauntlet Science 82, Gauntlet Ship 66 and Gauntlet Teach 81, 1.2 times the distance to 100 of the model above
  4. 4Claude Fable 5.1Claude Fable 5.1: Gauntlet Composite 74, the mean of Gauntlet Science 82, Gauntlet Ship 67 and Gauntlet Teach 73, 1.1 times the distance to 100 of the model above
  5. 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Composite 73, the mean of Gauntlet Science 86, Gauntlet Ship 55 and Gauntlet Teach 80, 1.0 times the distance to 100 of the model above
  6. 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Composite 72, the mean of Gauntlet Science 73, Gauntlet Ship 60 and Gauntlet Teach 82, 1.1 times the distance to 100 of the model above
  7. 7GPT-6.1 SolGPT-6.1 Sol: Gauntlet Composite 71, the mean of Gauntlet Science 81, Gauntlet Ship 64 and Gauntlet Teach 69, 1.0 times the distance to 100 of the model above
  8. 8GPT-6 AstraGPT-6 Astra: Gauntlet Composite 67, the mean of Gauntlet Science 85, Gauntlet Ship 40 and Gauntlet Teach 76, 1.1 times the distance to 100 of the model above
  9. 9Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Composite 56, the mean of Gauntlet Science 60, Gauntlet Ship 46 and Gauntlet Teach 62, 1.3 times the distance to 100 of the model above
  10. 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Composite 44, the mean of Gauntlet Science 59, Gauntlet Ship 40 and Gauntlet Teach 34, 1.3 times the distance to 100 of the model above

Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.

Gauntlet CompositeGauntlet ScienceGauntlet ShipGauntlet Teach

  1. 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Composite 81, the mean of Gauntlet Science 90, Gauntlet Ship 64 and Gauntlet Teach 91
  2. 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Composite 80, the mean of Gauntlet Science 89, Gauntlet Ship 71 and Gauntlet Teach 81
  3. 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Composite 76, the mean of Gauntlet Science 82, Gauntlet Ship 66 and Gauntlet Teach 81
  4. 4Claude Fable 5.1Claude Fable 5.1: Gauntlet Composite 74, the mean of Gauntlet Science 82, Gauntlet Ship 67 and Gauntlet Teach 73
  5. 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Composite 73, the mean of Gauntlet Science 86, Gauntlet Ship 55 and Gauntlet Teach 80
  6. 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Composite 72, the mean of Gauntlet Science 73, Gauntlet Ship 60 and Gauntlet Teach 82
  7. 7GPT-6.1 SolGPT-6.1 Sol: Gauntlet Composite 71, the mean of Gauntlet Science 81, Gauntlet Ship 64 and Gauntlet Teach 69
  8. 8GPT-6 AstraGPT-6 Astra: Gauntlet Composite 67, the mean of Gauntlet Science 85, Gauntlet Ship 40 and Gauntlet Teach 76
  9. 9Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Composite 56, the mean of Gauntlet Science 60, Gauntlet Ship 46 and Gauntlet Teach 62
  10. 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Composite 44, the mean of Gauntlet Science 59, Gauntlet Ship 40 and Gauntlet Teach 34

Not ranked, because not every benchmark ran them: Gemini 4 Argon (no Gauntlet Ship score), Gemini 4 Carbon (no Gauntlet Ship score), Gemini 4 Neon (no Gauntlet Ship score), GPT-6 Luna (no Gauntlet Ship score), Grok 4.7 (no Gauntlet Ship and Gauntlet Teach score) and Muse Wonder (no Gauntlet Ship and Gauntlet Teach score).

Each part comes from its own benchmark’s published run; nothing here is measured anew. The current runs of Gauntlet Science and Gauntlet Ship each fail one or more of their pre-registered checks, so those parts, and this ranking with them, are provisional.

Made from the published Gauntlet Science, Gauntlet Ship and Gauntlet Teach runs, newest graded 2026-11-20. Methodology

Gauntlet Teach measures whether a model can explain what it found so that someone else can use it. Each model gets the evidence for 12 invented Gauntlet Science worlds: the field records and the same designed experiments for every model. For each world it writes a plain-words explanation of the law it finds, at most 100 words, with no formula or math notation.

GPT-6.2 Astra sits 1.9× closer to 100 than Claude Sonnet 5.5, a paired gap of −8.2 points (95% −18.7 to +2.3).

  1. True lawreferenceTrue law, a reference script checked on every item: Gauntlet Teach score 98
  2. 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Teach score 91, its own Gauntlet Science score 86, check student 89, the leader, by a clear lead
  3. 2Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Teach score 82, its own Gauntlet Science score 71, check student 82, 1.9 times the distance to 100 of the model above, −8.2 points from the leader, 95% interval −18.7 to +2.3, not separated
  4. 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Teach score 81, its own Gauntlet Science score 78, check student 80, 1.1 times the distance to 100 of the model above, −9.5 points from the leader, 95% interval −26.3 to +7.4, not separated
  5. 4Claude Fable 5.2Claude Fable 5.2: Gauntlet Teach score 81, its own Gauntlet Science score 88, check student 80, 1.0 times the distance to 100 of the model above, −9.9 points from the leader, 95% interval −22.7 to +2.9, not separated
  6. 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Teach score 80, its own Gauntlet Science score 84, check student 79, 1.0 times the distance to 100 of the model above, −10.9 points from the leader, 95% interval −20.6 to −1.1, separated
  7. 6GPT-6 AstraGPT-6 Astra: Gauntlet Teach score 76, its own Gauntlet Science score 82, check student 71, 1.2 times the distance to 100 of the model above, −14.4 points from the leader, 95% interval −31.0 to +2.3, not separated
  8. 7Claude Fable 5.1Claude Fable 5.1: Gauntlet Teach score 73, its own Gauntlet Science score 75, check student 71, 1.1 times the distance to 100 of the model above, −17.3 points from the leader, 95% interval −32.2 to −2.4, separated
  9. 8Gemini 4 NeonGemini 4 Neon: Gauntlet Teach score 72, its own Gauntlet Science score 75, check student 72, 1.0 times the distance to 100 of the model above, −18.4 points from the leader, 95% interval −36.8 to +0.0, not separated
  10. 9GPT-6.1 SolGPT-6.1 Sol: Gauntlet Teach score 69, its own Gauntlet Science score 79, check student 66, 1.1 times the distance to 100 of the model above, −21.9 points from the leader, 95% interval −41.2 to −2.7, separated
  11. 10Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Teach score 62, its own Gauntlet Science score 64, check student 65, 1.2 times the distance to 100 of the model above, −28.9 points from the leader, 95% interval −48.4 to −9.3, separated
  12. 11Gemini 4 ArgonGemini 4 Argon: Gauntlet Teach score 58, its own Gauntlet Science score 68, check student 56, 1.1 times the distance to 100 of the model above, −33.0 points from the leader, 95% interval −54.6 to −11.4, separated
  13. 12Gemini 4 CarbonGemini 4 Carbon: Gauntlet Teach score 57, its own Gauntlet Science score 75, check student 57, 1.0 times the distance to 100 of the model above, −33.1 points from the leader, 95% interval −57.9 to −8.4, separated
  14. 13GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Teach score 34, its own Gauntlet Science score 61, check student 38, 1.5 times the distance to 100 of the model above, −56.2 points from the leader, 95% interval −81.5 to −31.0, separated
  15. 14GPT-6 LunaGPT-6 Luna: Gauntlet Teach score 17, its own Gauntlet Science score 45, check student 15, 1.3 times the distance to 100 of the model above, −73.1 points from the leader, 95% interval −91.7 to −54.6, separated
  16. No explanationreferenceNo explanation, a reference script checked on every item: Gauntlet Teach score 6

Placed by gap: equal distances are equal ratios of distance to 100, across only the models shown.

Gauntlet Teach score, with its 95% intervalThe model’s own Gauntlet Science score on the same worlds

  1. True lawreferenceTrue law, a reference row: Gauntlet Teach score 98, 95% interval 98 to 9995% 98–99
  2. 1GPT-6.2 Astra◇ Gauntlet Science 86Check student 89GPT-6.2 Astra: Gauntlet Teach score 91, 95% interval 85 to 96, its own Gauntlet Science score 86, check student 89, the leader95% 85–96Leader
  3. 2Claude Sonnet 5.5◇ Gauntlet Science 71Check student 82Claude Sonnet 5.5: Gauntlet Teach score 82, 95% interval 72 to 92, its own Gauntlet Science score 71, check student 82, −8.2 points from the leader, 95% interval −18.7 to +2.3, not separated95% 72–92−8.295% −18.7 to +2.3Not separated
  4. 3Claude Opus 5.5◇ Gauntlet Science 78Check student 80Claude Opus 5.5: Gauntlet Teach score 81, 95% interval 66 to 96, its own Gauntlet Science score 78, check student 80, −9.5 points from the leader, 95% interval −26.3 to +7.4, not separated95% 66–96−9.595% −26.3 to +7.4Not separated
  5. 4Claude Fable 5.2◇ Gauntlet Science 88Check student 80Claude Fable 5.2: Gauntlet Teach score 81, 95% interval 68 to 93, its own Gauntlet Science score 88, check student 80, −9.9 points from the leader, 95% interval −22.7 to +2.9, not separated95% 68–93−9.995% −22.7 to +2.9Not separated
  6. 5GPT-6.2 Sol◇ Gauntlet Science 84Check student 79GPT-6.2 Sol: Gauntlet Teach score 80, 95% interval 69 to 90, its own Gauntlet Science score 84, check student 79, −10.9 points from the leader, 95% interval −20.6 to −1.1, separated95% 69–90−10.995% −20.6 to −1.1● Separated
  7. 6GPT-6 Astra◇ Gauntlet Science 82Check student 71GPT-6 Astra: Gauntlet Teach score 76, 95% interval 61 to 92, its own Gauntlet Science score 82, check student 71, −14.4 points from the leader, 95% interval −31.0 to +2.3, not separated95% 61–92−14.495% −31.0 to +2.3Not separated
  8. 7Claude Fable 5.1◇ Gauntlet Science 75Check student 71Claude Fable 5.1: Gauntlet Teach score 73, 95% interval 59 to 88, its own Gauntlet Science score 75, check student 71, −17.3 points from the leader, 95% interval −32.2 to −2.4, separated95% 59–88−17.395% −32.2 to −2.4● Separated
  9. 8Gemini 4 Neon◇ Gauntlet Science 75Check student 72Gemini 4 Neon: Gauntlet Teach score 72, 95% interval 54 to 90, its own Gauntlet Science score 75, check student 72, −18.4 points from the leader, 95% interval −36.8 to 0.0, not separated95% 54–90−18.495% −36.8 to 0.0Not separated
  10. 9GPT-6.1 Sol◇ Gauntlet Science 79Check student 66GPT-6.1 Sol: Gauntlet Teach score 69, 95% interval 49 to 88, its own Gauntlet Science score 79, check student 66, −21.9 points from the leader, 95% interval −41.2 to −2.7, separated95% 49–88−21.995% −41.2 to −2.7● Separated
  11. 10Claude Haiku 5.5◇ Gauntlet Science 64Check student 65Claude Haiku 5.5: Gauntlet Teach score 62, 95% interval 44 to 80, its own Gauntlet Science score 64, check student 65, −28.9 points from the leader, 95% interval −48.4 to −9.3, separated95% 44–80−28.995% −48.4 to −9.3● Separated
  12. 11Gemini 4 Argon◇ Gauntlet Science 68Check student 56Gemini 4 Argon: Gauntlet Teach score 58, 95% interval 37 to 78, its own Gauntlet Science score 68, check student 56, −33.0 points from the leader, 95% interval −54.6 to −11.4, separated95% 37–78−33.095% −54.6 to −11.4● Separated
  13. 12Gemini 4 Carbon◇ Gauntlet Science 75Check student 57Gemini 4 Carbon: Gauntlet Teach score 57, 95% interval 34 to 81, its own Gauntlet Science score 75, check student 57, −33.1 points from the leader, 95% interval −57.9 to −8.4, separated95% 34–81−33.195% −57.9 to −8.4● Separated
  14. 13GPT-6.1 Terra◇ Gauntlet Science 61Check student 38GPT-6.1 Terra: Gauntlet Teach score 34, 95% interval 12 to 57, its own Gauntlet Science score 61, check student 38, −56.2 points from the leader, 95% interval −81.5 to −31.0, separated95% 12–57−56.295% −81.5 to −31.0● Separated
  15. 14GPT-6 Luna◇ Gauntlet Science 45Check student 15GPT-6 Luna: Gauntlet Teach score 17, 95% interval 2 to 32, its own Gauntlet Science score 45, check student 15, −73.1 points from the leader, 95% interval −91.7 to −54.6, separated95% 2–32−73.195% −91.7 to −54.6● Separated
  16. No explanationreferenceNo explanation, a reference row: Gauntlet Teach score 6, 95% interval 1 to 1095% 1–10

A fixed student, Claude Sonnet 5.5 at low effort, never sees the data. It reads each explanation and turns it into a formula; code computes that formula at the world’s held-out settings and scores the predictions with Gauntlet Science’s own scoring, from 0 to 100. No judge reads an explanation, and the student’s own knowledge cannot help, because the worlds were invented. Gemini 4 Argon reads the same explanations as a check student, and the two must rank the models alike. A world a model did not finish explaining in time is scored the way an empty explanation is, as the floor student; only a provider failure leaves a model unranked. Each score carries a 95% interval over the worlds, and a model’s gap to the leader is paired world by world. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are the same student without a model’s explanation. With no explanation it can only guess, so a model at that row taught it nothing. Handed the true law, it shows how far a perfect explanation could take it. The open diamond on each row is that model’s own Gauntlet Science score on the same 12 worlds, from Gauntlet Science run e05093a6, so a model that finds a law well but explains it poorly shows as a gap between the two.

Before any model ran, we fixed 5 checks the run must pass to count as a valid measurement (instrument, separation, student agreement, length and references reached an answer). This run passes all 5. One more figure is reported, not gated: not a copy of Gauntlet Science (lane-order Spearman with own Gauntlet Science score 0.69).

Not ranked, each for its own reason: Kimi (12 of 12 cells never reached an answer: the provider refused for rate or quota), Muse Wonder (5 of 12 cells never reached an answer: the provider refused for rate or quota on 1, and failed on 4 more) and Grok 4.7 (not run in this Gauntlet Teach run).

Worlds that ended at a time or output limit are scored as unanswered, which lowers a row: Gemini 4 Carbon on 3 of 12, Gemini 4 Neon on 2 of 12, Gemini 4 Argon on 2 of 12 and Claude Fable 5.2 on 1 of 12.

The worlds, the explanations and the students’ formulas stay private, so this page shows scores only.

Graded 2026-11-20; run 5775e008 on Gauntlet Science run e05093a6, 12 worlds, at most 100 words, protocol gauntlet-teach-v2. Full methodology

Gauntlet Judgment measures whether a model can find and fix a bug from the evidence alone. Each of the 63 items is one of Unsupervised Learning’s own tested source modules with two bugs planted in it. The model sees only the cleaned output of the failing tests and the folder’s file list with each file’s exported names, may open one file, and returns exact edits. Each model call has up to 15 minutes. Code applies them in a sandbox, and the hidden tests decide.

Gemini 4 Neon makes 1.2× fewer misses than GPT-6.1 Sol, a paired gap of −3.2 points (95% −12.0 to +5.7).

  1. Bugs revertedreferenceBugs reverted, a reference script checked on every item: Gauntlet Judgment score 100
  2. 1Gemini 4 NeonGemini 4 Neon: Gauntlet Judgment score 84, right file 92%, the leader
  3. 2GPT-6.1 SolGPT-6.1 Sol: Gauntlet Judgment score 81, right file 95%, 1.2 times the misses of the model above, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated
  4. 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Judgment score 81, right file 94%, 1.0 times the misses of the model above, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated
  5. 4Claude Fable 5.2Claude Fable 5.2: Gauntlet Judgment score 81, right file 95%, 1.0 times the misses of the model above, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated
  6. 5Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Judgment score 75, right file 94%, 1.3 times the misses of the model above, −9.5 points from the leader, 95% interval −18.1 to −1.0, separated
  7. 6Claude Fable 5.1Claude Fable 5.1: Gauntlet Judgment score 73, right file 89%, 1.1 times the misses of the model above, −11.1 points from the leader, 95% interval −21.1 to −1.1, separated
  8. 7Gemini 4 ArgonGemini 4 Argon: Gauntlet Judgment score 71, right file 95%, 1.1 times the misses of the model above, −12.7 points from the leader, 95% interval −22.1 to −3.3, separated
  9. 8Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Judgment score 70, right file 97%, 1.1 times the misses of the model above, −14.3 points from the leader, 95% interval −25.0 to −3.6, separated
  10. No editreferenceNo edit, a reference script checked on every item: Gauntlet Judgment score 0

Placed by gap: equal distances are equal ratios of misses, across only the models shown.

  1. Bugs revertedreferenceBugs reverted, a reference script checked on every item: Gauntlet Judgment score 100Every item
  2. 1Gemini 4 NeonRight file 92%Gemini 4 Neon: Gauntlet Judgment score 84, 95% interval 75 to 93, right file 92%, the leader95% 75–93Leader
  3. 2GPT-6.1 SolRight file 95%GPT-6.1 Sol: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 95%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
  4. 3Claude Opus 5.5Right file 94%Claude Opus 5.5: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 94%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
  5. 4Claude Fable 5.2Right file 95%Claude Fable 5.2: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 95%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
  6. 5Claude Sonnet 5.5Right file 94%Claude Sonnet 5.5: Gauntlet Judgment score 75, 95% interval 64 to 85, right file 94%, −9.5 points from the leader, 95% interval −18.1 to −1.0, separated95% 64–85−9.595% −18.1 to −1.0● Separated
  7. 6Claude Fable 5.1Right file 89%Claude Fable 5.1: Gauntlet Judgment score 73, 95% interval 62 to 84, right file 89%, −11.1 points from the leader, 95% interval −21.1 to −1.1, separated95% 62–84−11.195% −21.1 to −1.1● Separated
  8. 7Gemini 4 ArgonRight file 95%Gemini 4 Argon: Gauntlet Judgment score 71, 95% interval 60 to 83, right file 95%, −12.7 points from the leader, 95% interval −22.1 to −3.3, separated95% 60–83−12.795% −22.1 to −3.3● Separated
  9. 8Claude Haiku 5.5Right file 97%Claude Haiku 5.5: Gauntlet Judgment score 70, 95% interval 58 to 81, right file 97%, −14.3 points from the leader, 95% interval −25.0 to −3.6, separated95% 58–81−14.395% −25.0 to −3.6● Separated
  10. No editreferenceNo edit, a reference script checked on every item: Gauntlet Judgment score 0Every item

An item is fixed when every hidden test passes after the edits. Code makes every number: no judge reads the work. The Gauntlet Judgment score is the share of items fixed, with a 95% interval over the items; the gap to the leader is paired item by item. Right file is the share of items where the file the model opened was the one holding the bugs. Every model runs at high effort; a model whose API has no effort setting runs at its own default. The two grey rows are scripts, not models, checked on every item: no edit fixes nothing, and reverting the planted bugs fixes everything, so every item counted can be fixed.

Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (instrument, no cheap twin, separation, headroom, stability, length and reproducibility). This run fails 2 (stability and reproducibility), so read its ranking as provisional.

Two companion floor checks run too: Brake, refusing a destructive command when the deciding fact is buried in tool output, and Receipt, reading a requirement out of a long log. Every model scores 100 on both so far, so they do not separate models and are not ranked.

The items are private code, so this page shows counts and scores only: never an item’s module, its code, the planted bugs, the failing output or a model’s reply.

Graded 2026-11-20; run e73a52c0, item set v11, protocol gauntlet-judgment-patch-v2. Not part of Gauntlet Composite. Full methodology

Each model is one dot: the plain mean, on one 0–100 scale, of every part switched on above that it has: its Artificial Analysis Intelligence Index averaged over its efforts, and its Gauntlet Science, Gauntlet Ship and Gauntlet Teach scores, so each counts equally. A model with no Gauntlet score switched on is left out. Cost is Artificial Analysis’s cost per index task, averaged over the same efforts. It shows the models picked in the Models list above. Hover a mark for every part.