ai benchmarks

Real-world AI performance metrics

Gauntlet Composite updated

Gauntlet Science updated

Gauntlet Ship updated

Gauntlet Teach updated

Gauntlet Judgment updated

Artificial Analysis data updated

  1. 1GPT-6.2 AstraGPT-6.2 Astra: Gauntlet Composite 81, the mean of Gauntlet Science 90, Gauntlet Ship 64 and Gauntlet Teach 91
  2. 2Claude Fable 5.2Claude Fable 5.2: Gauntlet Composite 80, the mean of Gauntlet Science 89, Gauntlet Ship 71 and Gauntlet Teach 81
  3. 3Claude Opus 5.5Claude Opus 5.5: Gauntlet Composite 76, the mean of Gauntlet Science 82, Gauntlet Ship 66 and Gauntlet Teach 81
  4. 4Claude Fable 5.1Claude Fable 5.1: Gauntlet Composite 74, the mean of Gauntlet Science 82, Gauntlet Ship 67 and Gauntlet Teach 73
  5. 5GPT-6.2 SolGPT-6.2 Sol: Gauntlet Composite 73, the mean of Gauntlet Science 86, Gauntlet Ship 55 and Gauntlet Teach 80
  6. 6Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Composite 72, the mean of Gauntlet Science 73, Gauntlet Ship 60 and Gauntlet Teach 82
  7. 7GPT-6.1 SolGPT-6.1 Sol: Gauntlet Composite 71, the mean of Gauntlet Science 81, Gauntlet Ship 64 and Gauntlet Teach 69
  8. 8GPT-6 AstraGPT-6 Astra: Gauntlet Composite 67, the mean of Gauntlet Science 85, Gauntlet Ship 40 and Gauntlet Teach 76
  9. 9Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Composite 56, the mean of Gauntlet Science 60, Gauntlet Ship 46 and Gauntlet Teach 62
  10. 10GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Composite 44, the mean of Gauntlet Science 59, Gauntlet Ship 40 and Gauntlet Teach 34

About AI Benchmarks

What each AI model scores, and what that score costs, at every reasoning effort. The Curves view draws each model’s effort levels as one line of score against cost. The UL Gauntlet tab holds our own benchmarks: Gauntlet Science ranks how well each model does science on worlds it has never seen. Gauntlet Ship ranks how often each model ships real work in a codebase it has never seen. Gauntlet Teach ranks how well each model explains what it found, by how well a student that never saw the data can predict from its explanation. Gauntlet Composite, the tab’s first view, is their average. Gauntlet Judgment ranks how often each model fixes real code with two planted bugs from the failing tests’ output alone; it is not part of that average yet.

Where the idea came from

In his video “Anthropic Actually Fixed Opus”, Theo (t3.gg) showed a chart that drew each model’s reasoning-effort levels as one curve of Intelligence Index against cost per task. It made the trade-off between paying more and getting a smarter answer easy to see. The Curves view is that chart.

Where the data comes from

Each benchmark carries its owner’s mark, so you can always tell whose numbers you are reading. Two sources feed the site’s own views; the Other benchmarks tab adds respected outside ones, each credited where it appears.

Artificial Analysis’s free Data API is read every hour into a snapshot, and the last good snapshot feeds the Curves view and History, which records what changed. Unsupervised Learning’s own benchmarks publish scores only, never answers, which reach the site at each deploy and feed the UL Gauntlet views: a ranking per benchmark, and Gauntlet Composite, their average.

Two sources on separate rails. History records every change in Artificial Analysis’s data.

Artificial Analysis

The Intelligence, Coding and Agentic indexes and their cost per task come from Artificial Analysis, which benchmarks AI models independently. We read it through their free Data API, never by scraping their pages. Their free tier asks for credit wherever the data appears, so every page names them in its footer.

The data refreshes every hour, on the hour. Each refresh reads the full model list, checks that it has the shape we expect, and joins each model’s effort settings into one line. If a refresh fails, or the new list has lost more than half of its data points, we keep showing the last good copy instead.

Whenever a refresh brings different numbers, the History view records every score, price and release that changed, with a short AI-written summary of what moved.

The data on this page was last updated .

Unsupervised Learning’s Gauntlet Science, Gauntlet Ship, Gauntlet Teach and Gauntlet Judgment

The UL Gauntlet tab holds our own benchmarks. Each is scored by code, with no judge reading an answer, and each publishes its scores, never the answers or the private material behind them.

Gauntlet Science, the first, sets every model the same 24 worlds, generated fresh for each run so none can have been seen before. In each world the model runs 6 experiments of its own design, then predicts past anything it measured, and code that knows the world’s hidden law scores every prediction. No judge reads an answer. These scores are from ; its ranking and the section on Gauntlet Science below say more.

Gauntlet Ship, the second, gives every model the same 55 real tasks from our own working sessions, each in a codebase it has never seen, with the working conversation as its ask. Code runs hidden checks written from that conversation and decides whether each change ships. No judge reads the work. These scores are from ; its ranking and the section on Gauntlet Ship below say more.

Gauntlet Teach, the third, gives every model the evidence for 12 invented Gauntlet Science worlds and asks for a plain-words explanation of each law, at most 100 words with no formula. A student model that never saw the data predicts from each explanation alone, and code scores the predictions. No judge reads the explanations. These scores are from ; its ranking and the section on Gauntlet Teach below say more.

Gauntlet Judgment, the fourth, gives every model the same 63 items, each one of our own tested source modules with two bugs planted in it, and only the failing tests’ output and the folder’s file list to go on. Code applies the model’s edits and the hidden tests decide whether each item is fixed. No judge reads the work. Its ranking is provisional, because its run has not passed every check of its validity gate. It is not part of Gauntlet Composite. These scores are from ; its ranking and the section on Gauntlet Judgment below say more.

Gauntlet Composite, which the UL Gauntlet tab opens on, puts these benchmarks together: each model’s plain mean of its Gauntlet Science, Gauntlet Ship and Gauntlet Teach scores, on 0 to 100. Every model runs at high effort; a model whose API has no effort setting runs at its own default. Only the 10 models every one of them ran are ranked, so every dot is made of the same parts. It measures nothing anew; the ranking at the top of this page is its view.

What the numbers mean

Intelligence Index
Artificial Analysis’s combined score across its set of evaluations (version 5 in the data shown here). Their methodology page lists what goes into it.
Coding Index and Agentic Index
Narrower scores from Artificial Analysis, each an equal-weighted average of the evaluations in its area. Their capability indices page defines both.
Cost per task
What Artificial Analysis measured it cost, in US dollars, to run one Intelligence Index task at that setting. We use the same cost beside Coding and Agentic scores, so a setting costs the same whichever score you rank by.
Effort level
Many models let you choose how much they reason before they answer, usually named low, medium, high, xhigh or max. Artificial Analysis tests each setting as its own entry. We join the settings of one release into one line, lowest effort first. More effort usually scores higher and costs more.
lowmedhighxhmaxover budget
Dots grow with effort, from low to max.
Why some models have fewer levels
A model shows only the settings its maker offers and Artificial Analysis has tested. A model with one setting is a single dot.
Why new models can lack Coding or Agentic scores
The newest releases can have an Intelligence Index score before Artificial Analysis publishes their Coding or Agentic scores. Until it does, we list them as not yet scored, never as zero.

What Gauntlet Science measures

Gauntlet Science is Unsupervised Learning’s first benchmark, and its ranking scores each model from 0 to 100. It asks whether a model can find out something new for itself: look at a world it has never seen, test it, work out the rule underneath, and predict what happens where it has never looked.

Most science benchmarks ask about science that is already written down, and a model can learn what is written down. Here the thing to be discovered did not exist until the run began, so a high score cannot come from memory.

Worlds generated fresh, so nothing can be memorised

Every run generates its worlds from a new seed. Each world is a made-up measurable system, a handful of quantities and an outcome, joined by a hidden law. Some quantities matter and some are decoys. This run used 24 worlds from seed 20261112, drawn from 6 families of law on three rungs of difficulty:

Family of lawRungWorlds in this run
additiveRung 1, the simplest4
multiplicativeRung 1, the simplest4
gatedRung 24
ratioRung 24
hidden-regimeRung 3, the hardest4
peakRung 3, the hardest4

Six experiments, then predictions beyond the data

  1. The model reads a set of noisy observations from the world, all inside the range anyone has measured.
  2. It designs its own experiments, 6 in all, choosing the settings it wants to try, and reads back what the world does.
  3. It states the law it found, the strongest rival law, and the experiment that would tell them apart.
  4. It makes 20 predictions, each with an interval it is 80% sure of. Half fall inside the measured range and half fall beyond anything it measured, where only the right law predicts well.
  5. The test is closed-book. The model has no tools, no code and no web access.

Scored by code against the hidden law, with no judge

Code wrote each world, so code knows its law and scores every prediction against it with an interval score: a tight interval around the true value scores well, a wide one or a miss scores badly. A score of 100 is every prediction exact and certain; 0 is a miss or no answer. The headline score counts the predictions beyond the data. No model reads or grades another model’s answer, so there is no judge’s taste to correct for.

Each model’s score is its mean over the worlds, with a 95% interval. Its gap to the leader is paired world by world, and the ranking marks a gap as separated only when its whole 95% interval sits below zero.

The floor and ceiling scripts

Two scripts play every world beside the models, as reference rows. The floor script runs random experiments and fits a straight line: a model below it did no better than that. The ceiling script spreads its experiments to the edges of the range and picks the best-fitting law from the families the worlds are drawn from: a practical top, though not a perfect one. In this run the floor scored 37 and the ceiling 93.

The validity gate

Before any model ran, we fixed the checks a run has to pass to count as a measurement of anything, among them that it separates the models, that the hardest worlds leave headroom, that the ranking holds up when the worlds are split in half, that longer answers do not score better for being long, and that the leader beats the floor script. A run that fails is reported as provisional. This run fails 1 of the 7 checks (stability), so its ranking is provisional.

CheckResultThis run
SeparationPassedleader-to-bottom spread 45.5 vs leader half-width 6.3
HeadroomPassedleader rung-3 mean 83.8 (must be below 85)
StabilityFailedfamily-stratified split-half Spearman 0.79, mean of 200 splits (needs 0.80)
LengthPassed|r| between reply length and world score, centred by lane and world, 0.06 (must be below 0.50)
CapPassedSpearman of capped and uncapped lane orders 0.90 (needs 0.80)
No cheap twinPassedleader minus floor 52.9 [43.0, 62.9] (must lie above 0)
EffortPassed16 models in 16 lanes (one lane a model); each has exactly one cell per world

The Gauntlet Science ranking

Each row is one model, best first: its score from 0 to 100, its 95% interval, and its gap to the leader with that gap’s interval. The grey rows are the floor and ceiling scripts.

  1. Ceiling scriptreferenceCeiling script, a reference script: score 93, 95% interval 90 to 9795% 90–97
  2. 1GPT-6.2 AstraGPT-6.2 Astra: score 90, 95% interval 83 to 96, the leader95% 83–96Leader
  3. 2Claude Fable 5.2Claude Fable 5.2: score 89, 95% interval 84 to 94, −0.5 points from the leader, 95% interval −3.6 to +2.6, not separated95% 84–94−0.595% −3.6 to +2.6Not separated
  4. 3GPT-6.2 SolGPT-6.2 Sol: score 86, 95% interval 79 to 92, −3.8 points from the leader, 95% interval −5.6 to −1.9, separated95% 79–92−3.895% −5.6 to −1.9● Separated
  5. 4GPT-6 AstraGPT-6 Astra: score 85, 95% interval 79 to 92, −4.3 points from the leader, 95% interval −7.0 to −1.7, separated95% 79–92−4.395% −7.0 to −1.7● Separated
  6. 5Claude Opus 5.5Claude Opus 5.5: score 82, 95% interval 74 to 91, −7.1 points from the leader, 95% interval −11.5 to −2.7, separated95% 74–91−7.195% −11.5 to −2.7● Separated
  7. 6Claude Fable 5.1Claude Fable 5.1: score 82, 95% interval 74 to 91, −7.4 points from the leader, 95% interval −12.3 to −2.5, separated95% 74–91−7.495% −12.3 to −2.5● Separated
  8. 7Gemini 4 NeonGemini 4 Neon: score 81, 95% interval 69 to 93, −8.6 points from the leader, 95% interval −21.5 to +4.2, not separated95% 69–93−8.695% −21.5 to +4.2Not separated
  9. 8GPT-6.1 SolGPT-6.1 Sol: score 81, 95% interval 73 to 88, −8.8 points from the leader, 95% interval −14.6 to −3.1, separated95% 73–88−8.895% −14.6 to −3.1● Separated
  10. 9Gemini 4 CarbonGemini 4 Carbon: score 78, 95% interval 65 to 91, −11.5 points from the leader, 95% interval −24.4 to +1.5, not separated95% 65–91−11.595% −24.4 to +1.5Not separated
  11. 10Muse WonderMuse Wonder: score 78, 95% interval 69 to 87, −11.7 points from the leader, 95% interval −17.7 to −5.6, separated95% 69–87−11.795% −17.7 to −5.6● Separated
  12. 11Claude Sonnet 5.5Claude Sonnet 5.5: score 73, 95% interval 65 to 81, −16.4 points from the leader, 95% interval −22.2 to −10.6, separated95% 65–81−16.495% −22.2 to −10.6● Separated
  13. 12Gemini 4 ArgonGemini 4 Argon: score 66, 95% interval 55 to 77, −23.6 points from the leader, 95% interval −34.4 to −12.8, separated95% 55–77−23.695% −34.4 to −12.8● Separated
  14. 13Grok 4.7Grok 4.7: score 64, 95% interval 51 to 77, −25.5 points from the leader, 95% interval −36.0 to −15.0, separated95% 51–77−25.595% −36.0 to −15.0● Separated
  15. 14Claude Haiku 5.5Claude Haiku 5.5: score 60, 95% interval 50 to 69, −29.6 points from the leader, 95% interval −38.5 to −20.7, separated95% 50–69−29.695% −38.5 to −20.7● Separated
  16. 15GPT-6.1 TerraGPT-6.1 Terra: score 59, 95% interval 48 to 69, −30.9 points from the leader, 95% interval −41.9 to −19.8, separated95% 48–69−30.995% −41.9 to −19.8● Separated
  17. 16GPT-6 LunaGPT-6 Luna: score 44, 95% interval 33 to 55, −45.5 points from the leader, 95% interval −58.7 to −32.2, separated95% 33–55−45.595% −58.7 to −32.2● Separated
  18. Floor scriptreferenceFloor script, a reference script: score 37, 95% interval 26 to 4795% 26–47

Not ranked, because a model is ranked only when it finished every world: Kimi (2 of 24 cells never reached an answer: the provider failed).

Gauntlet Science’s scores come from protocol gauntlet-discovery-v3, run f8b224a2, made on . Scores from different protocols are never compared.

What Gauntlet Ship measures

Gauntlet Ship is Unsupervised Learning’s second benchmark, and its ranking scores each model by the share of real tasks it ships. Gauntlet Science asks whether a model can find something out. Gauntlet Ship asks whether it can do the work: take a request from a real conversation, in code it has never seen, and leave a change that does what was asked.

Real work, in a codebase the model has never seen

Every item is a real task from Unsupervised Learning’s own working sessions: a conversation that asked for a change, and the change that was accepted. This run has 55 items across 16 parts of the codebase. The model gets the working conversation as its ask and the repository as it stood before the fix, and works through one plain tool loop, reading files, writing files and running tests, for at most 30 steps. Then it says whether it is done.

Hidden checks, scored by code, with no judge

Each item’s checks are written from its conversation and hidden from the model. A check is kept only when it fails on the code before the accepted fix and passes after it, and the checks are screened so they test nothing the conversation left open. An item is shipped when every hidden check passes and no test that passed before breaks. Code runs the checks and makes every number; no model reads or grades another model’s work.

The Gauntlet Ship score is the share of items shipped, with a 95% interval over the items. A model’s gap to the leader is paired item by item, and the ranking marks it separated only when its whole 95% interval sits below zero. Beside the score: hidden checks, the share of hidden checks passed; false done, the share of “done” claims that did not ship; right files, how far the files it changed overlap the accepted fix’s; and the safeguards its changes weakened, such as a deleted or loosened test.

Two scripts play every item beside the models, as reference rows. No change finishes at once, and shipped 0% of the items. The accepted fix writes the change that was actually accepted, and shipped 100%; an item it cannot ship is dropped before scoring, so every item counted can be shipped.

The validity gate

Before any model ran, we fixed the checks a run has to pass to count as a measurement of anything: among them that every kept item is valid, that the models separate, that the leader leaves headroom, that the ranking holds when the items are split in half, that longer transcripts do not ship more for being long, and that the leader beats no change. A run that fails any of them is shown as provisional. This run fails 1 of the 7 checks (stability), so its ranking is provisional.

CheckResultThis run
Items validPassed55 kept, 0 dropped; accepted fix ships on every kept item; floor ships on none
SeparationPassedtop-to-bottom spread 30.9% vs leader half-width 12.1%
HeadroomPassedleader ships 70.9% (must be under 85%)
StabilityFailedodd-even Spearman 0.73 (needs 0.80)
No cheap twinPassedleader minus floor 70.9% [58.8%, 83.0%] (must lie above 0)
LengthPassed|r| between transcript length and outcome, centred by lane and item, 0.15 (must be below 0.50)
EffortPassed10 models in 10 lanes (one lane a model); each has exactly one cell per item

Argon ran but is left out of this ranking: 47 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked. Carbon ran but is left out of this ranking: 54 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked. Neon ran but is left out of this ranking: 50 of its 55 items ended at the bench's output or time limit rather than on its work; it is replayed under longer limits before it is ranked.

The Gauntlet Ship ranking

Each row is one model, best first: its Gauntlet Ship score, its 95% interval, its hidden-check and false-done shares, and its gap to the leader with that gap’s interval. The grey rows are no change and the accepted fix.

  1. Accepted fixreferenceAccepted fix, a reference script: Gauntlet Ship score 100, 95% interval 100 to 10095% 100–100
  2. 1Claude Fable 5.2Hidden checks 83%False done 19%Claude Fable 5.2: Gauntlet Ship score 71, 95% interval 59 to 83, hidden checks 83%, false done 19%, the leader95% 59–83Leader
  3. 2Claude Fable 5.1Hidden checks 77%False done 30%Claude Fable 5.1: Gauntlet Ship score 67, 95% interval 55 to 80, hidden checks 77%, false done 30%, −3.6 points from the leader, 95% interval −15.0 to +7.7, not separated95% 55–80−3.695% −15.0 to +7.7Not separated
  4. 3Claude Opus 5.5Hidden checks 74%False done 26%Claude Opus 5.5: Gauntlet Ship score 66, 95% interval 53 to 78, hidden checks 74%, false done 26%, −5.5 points from the leader, 95% interval −17.3 to +6.4, not separated95% 53–78−5.595% −17.3 to +6.4Not separated
  5. 4GPT-6.1 SolHidden checks 76%False done 30%GPT-6.1 Sol: Gauntlet Ship score 64, 95% interval 51 to 77, hidden checks 76%, false done 30%, −7.3 points from the leader, 95% interval −19.6 to +5.0, not separated95% 51–77−7.395% −19.6 to +5.0Not separated
  6. 5GPT-6.2 AstraHidden checks 76%False done 31%GPT-6.2 Astra: Gauntlet Ship score 64, 95% interval 51 to 77, hidden checks 76%, false done 31%, −7.3 points from the leader, 95% interval −18.5 to +3.9, not separated95% 51–77−7.395% −18.5 to +3.9Not separated
  7. 6Claude Sonnet 5.5Hidden checks 72%False done 39%Claude Sonnet 5.5: Gauntlet Ship score 60, 95% interval 47 to 73, hidden checks 72%, false done 39%, −10.9 points from the leader, 95% interval −21.9 to +0.1, not separated95% 47–73−10.995% −21.9 to +0.1Not separated
  8. 7GPT-6.2 SolHidden checks 68%False done 40%GPT-6.2 Sol: Gauntlet Ship score 55, 95% interval 41 to 68, hidden checks 68%, false done 40%, −16.4 points from the leader, 95% interval −26.2 to −6.5, separated95% 41–68−16.495% −26.2 to −6.5● Separated
  9. 8Claude Haiku 5.5Hidden checks 61%False done 46%Claude Haiku 5.5: Gauntlet Ship score 46, 95% interval 32 to 59, hidden checks 61%, false done 46%, −25.5 points from the leader, 95% interval −38.1 to −12.8, separated95% 32–59−25.595% −38.1 to −12.8● Separated
  10. 9GPT-6 AstraHidden checks 55%False done 35%GPT-6 Astra: Gauntlet Ship score 40, 95% interval 27 to 53, hidden checks 55%, false done 35%, −30.9 points from the leader, 95% interval −44.2 to −17.6, separated95% 27–53−30.995% −44.2 to −17.6● Separated
  11. 10GPT-6.1 TerraHidden checks 52%False done 40%GPT-6.1 Terra: Gauntlet Ship score 40, 95% interval 27 to 53, hidden checks 52%, false done 40%, −30.9 points from the leader, 95% interval −45.2 to −16.6, separated95% 27–53−30.995% −45.2 to −16.6● Separated
  12. No changereferenceNo change, a reference script: Gauntlet Ship score 0, 95% interval 0 to 095% 0–0
ModelShippedHidden checksFalse doneRight filesStepsWeakened
Claude Fable 5.271%83%19%57%17.68
Claude Fable 5.167%77%30%54%15.56
Claude Opus 5.566%74%26%52%14.63
GPT-6.1 Sol64%76%30%46%23.324
GPT-6.2 Astra64%76%31%47%21.430
Claude Sonnet 5.560%72%39%51%14.98
GPT-6.2 Sol55%68%40%49%23.314
Claude Haiku 5.546%61%46%43%8.55
GPT-6 Astra40%55%35%34%15.520
GPT-6.1 Terra40%52%40%31%15.88

Gauntlet Ship’s scores come from protocol gauntlet-ship-v1, item set v4, run f401c811, made on . Scores from different protocols or item sets are never compared. The items are private work, so this page shows counts and scores only: never an item’s conversation, its code, its checks or a model’s transcript.

What Gauntlet Teach measures

Gauntlet Teach is Unsupervised Learning’s third benchmark, and its ranking scores each model by how well someone who never saw the data can predict from its explanation. Gauntlet Science asks whether a model can find a law for itself. Gauntlet Teach asks whether it can then say what it found so plainly that another mind can use it.

A plain-words explanation of an invented world

Each model gets the evidence for 12 Gauntlet Science worlds: the field records, and the same lab experiments, designed by code, for every model, so no model is helped or hurt by experiments of its own. For each world it writes how the outcome works for a reader who will never see the records: which quantities matter, which do not, and the numbers the reader needs. Each explanation is at most 100 words, in plain words, with no formula, no equation and no math notation.

A student that never saw the data, scored by code

A fixed student, Claude Sonnet 5.5 at low effort, reads each explanation alone and writes the law as a formula. Code computes that formula at the world’s held-out settings, up to twice the recorded ranges, and scores the predictions with Gauntlet Science’s own scoring, from 0 to 100. No judge reads an explanation. The worlds were invented for the test, so nothing the student already knows can stand in for a good explanation.

Gemini 4 Argon reads the same explanations as a check student, and the gate requires the two students to rank the models alike, so a score is not one reader’s quirk. Each model’s Gauntlet Teach score is the headline student’s mean over the worlds, with a 95% interval. Its gap to the leader is paired world by world, and the ranking marks it separated only when its whole 95% interval sits below zero. A world a model did not finish explaining in time is scored the way an empty explanation is, as the floor student; only a provider failure leaves a model unranked. Beside each score the ranking shows the model’s own Gauntlet Science score on the same worlds: a model can find a law well and explain it poorly.

Two reference rows bracket every model. With no explanation the student scored 6; handed the true law, it scored 98.

The validity gate

Before any model ran, we fixed the checks a run has to pass to count as a measurement of anything: that the true law beats no explanation by a clear margin, that the models separate, that the two students rank the models alike, and that longer explanations do not score better for being long. One more figure, how closely the order follows each model’s own Gauntlet Science score, is reported but not gated. A run that fails a check is shown as provisional. This run passes all 5 checks.

CheckResultThis run
InstrumentPassedceiling minus floor 92.8 [88.3, 97.3]
SeparationPassedspread 73.1 vs leader half-width 5.1
Student agreementPassedSpearman 0.98 between Claude Sonnet 5.5 at low effort and Gemini 4 Argon
LengthPassedcentred rank correlation of words with score 0.11
Not a copy of Gauntlet ScienceReportedlane-order Spearman with own Gauntlet Science score 0.69
References reached an answerPassedevery floor and ceiling cell answered

The Gauntlet Teach ranking

Each row is one model, best first: its Gauntlet Teach score, its 95% interval, its own Gauntlet Science score as an open diamond, the check student’s score, and its gap to the leader with that gap’s interval. The grey rows are no explanation and the true law.

  1. True lawreferenceTrue law, a reference row: Gauntlet Teach score 98, 95% interval 98 to 9995% 98–99
  2. 1GPT-6.2 Astra◇ Gauntlet Science 86Check student 89GPT-6.2 Astra: Gauntlet Teach score 91, 95% interval 85 to 96, its own Gauntlet Science score 86, check student 89, the leader95% 85–96Leader
  3. 2Claude Sonnet 5.5◇ Gauntlet Science 71Check student 82Claude Sonnet 5.5: Gauntlet Teach score 82, 95% interval 72 to 92, its own Gauntlet Science score 71, check student 82, −8.2 points from the leader, 95% interval −18.7 to +2.3, not separated95% 72–92−8.295% −18.7 to +2.3Not separated
  4. 3Claude Opus 5.5◇ Gauntlet Science 78Check student 80Claude Opus 5.5: Gauntlet Teach score 81, 95% interval 66 to 96, its own Gauntlet Science score 78, check student 80, −9.5 points from the leader, 95% interval −26.3 to +7.4, not separated95% 66–96−9.595% −26.3 to +7.4Not separated
  5. 4Claude Fable 5.2◇ Gauntlet Science 88Check student 80Claude Fable 5.2: Gauntlet Teach score 81, 95% interval 68 to 93, its own Gauntlet Science score 88, check student 80, −9.9 points from the leader, 95% interval −22.7 to +2.9, not separated95% 68–93−9.995% −22.7 to +2.9Not separated
  6. 5GPT-6.2 Sol◇ Gauntlet Science 84Check student 79GPT-6.2 Sol: Gauntlet Teach score 80, 95% interval 69 to 90, its own Gauntlet Science score 84, check student 79, −10.9 points from the leader, 95% interval −20.6 to −1.1, separated95% 69–90−10.995% −20.6 to −1.1● Separated
  7. 6GPT-6 Astra◇ Gauntlet Science 82Check student 71GPT-6 Astra: Gauntlet Teach score 76, 95% interval 61 to 92, its own Gauntlet Science score 82, check student 71, −14.4 points from the leader, 95% interval −31.0 to +2.3, not separated95% 61–92−14.495% −31.0 to +2.3Not separated
  8. 7Claude Fable 5.1◇ Gauntlet Science 75Check student 71Claude Fable 5.1: Gauntlet Teach score 73, 95% interval 59 to 88, its own Gauntlet Science score 75, check student 71, −17.3 points from the leader, 95% interval −32.2 to −2.4, separated95% 59–88−17.395% −32.2 to −2.4● Separated
  9. 8Gemini 4 Neon◇ Gauntlet Science 75Check student 72Gemini 4 Neon: Gauntlet Teach score 72, 95% interval 54 to 90, its own Gauntlet Science score 75, check student 72, −18.4 points from the leader, 95% interval −36.8 to 0.0, not separated95% 54–90−18.495% −36.8 to 0.0Not separated
  10. 9GPT-6.1 Sol◇ Gauntlet Science 79Check student 66GPT-6.1 Sol: Gauntlet Teach score 69, 95% interval 49 to 88, its own Gauntlet Science score 79, check student 66, −21.9 points from the leader, 95% interval −41.2 to −2.7, separated95% 49–88−21.995% −41.2 to −2.7● Separated
  11. 10Claude Haiku 5.5◇ Gauntlet Science 64Check student 65Claude Haiku 5.5: Gauntlet Teach score 62, 95% interval 44 to 80, its own Gauntlet Science score 64, check student 65, −28.9 points from the leader, 95% interval −48.4 to −9.3, separated95% 44–80−28.995% −48.4 to −9.3● Separated
  12. 11Gemini 4 Argon◇ Gauntlet Science 68Check student 56Gemini 4 Argon: Gauntlet Teach score 58, 95% interval 37 to 78, its own Gauntlet Science score 68, check student 56, −33.0 points from the leader, 95% interval −54.6 to −11.4, separated95% 37–78−33.095% −54.6 to −11.4● Separated
  13. 12Gemini 4 Carbon◇ Gauntlet Science 75Check student 57Gemini 4 Carbon: Gauntlet Teach score 57, 95% interval 34 to 81, its own Gauntlet Science score 75, check student 57, −33.1 points from the leader, 95% interval −57.9 to −8.4, separated95% 34–81−33.195% −57.9 to −8.4● Separated
  14. 13GPT-6.1 Terra◇ Gauntlet Science 61Check student 38GPT-6.1 Terra: Gauntlet Teach score 34, 95% interval 12 to 57, its own Gauntlet Science score 61, check student 38, −56.2 points from the leader, 95% interval −81.5 to −31.0, separated95% 12–57−56.295% −81.5 to −31.0● Separated
  15. 14GPT-6 Luna◇ Gauntlet Science 45Check student 15GPT-6 Luna: Gauntlet Teach score 17, 95% interval 2 to 32, its own Gauntlet Science score 45, check student 15, −73.1 points from the leader, 95% interval −91.7 to −54.6, separated95% 2–32−73.195% −91.7 to −54.6● Separated
  16. No explanationreferenceNo explanation, a reference row: Gauntlet Teach score 6, 95% interval 1 to 1095% 1–10
ModelGauntlet TeachCheck studentGauntlet Science, same worldsMean words
GPT-6.2 Astra91898664
Claude Sonnet 5.582827192
Claude Opus 5.581807894
Claude Fable 5.281808896
GPT-6.2 Sol80798458
GPT-6 Astra76718265
Claude Fable 5.173717598
Gemini 4 Neon72727583
GPT-6.1 Sol69667961
Claude Haiku 5.562656493
Gemini 4 Argon58566866
Gemini 4 Carbon57577584
GPT-6.1 Terra34386167
GPT-6 Luna17154569

Not ranked, each for its own reason: Kimi (12 of 12 cells never reached an answer: the provider refused for rate or quota), Muse Wonder (5 of 12 cells never reached an answer: the provider refused for rate or quota on 1, and failed on 4 more) and Grok 4.7 (not run in this Gauntlet Teach run).

Gauntlet Teach’s scores come from protocol gauntlet-teach-v2, run 5775e008 on Gauntlet Science run e05093a6, made on . Scores from different protocols are never compared. The worlds, the explanations and the students’ formulas stay private, so this page shows scores only.

What Gauntlet Judgment measures

Gauntlet Judgment is Unsupervised Learning’s fourth benchmark, and its ranking scores each model by the share of broken modules it fixes. It asks whether a model can read the evidence a failing test leaves, decide where the fault is, and repair it without being shown the answer.

Real code with two planted bugs, and only the failing output to go on

Each item is one of Unsupervised Learning’s own tested source modules with two bugs planted in it, each of which alone breaks a hidden test that the other leaves passing, and whose failing output names the file the bugs are in or one of its exports. This run has 63 items. The model sees only the cleaned output of the failing tests and the folder’s file list with each file’s exported names, may open one file, and returns exact edits. Each model call has up to 15 minutes.

The hidden tests decide, with no judge

Code applies the edits in a sandbox, restores the hidden tests and runs them. An item is fixed only when every test passes. Code makes every number; no model reads or grades another model’s work.

The Gauntlet Judgment score is the share of items fixed, with a 95% interval over the items. A model’s gap to the leader is paired item by item, and the ranking marks it separated only when its whole 95% interval sits below zero. Beside the score, right file is the share of items where the file the model opened was the one holding the bugs.

Two scripts are checked on every item as reference rows: no edit fixed 0% of the items, and reverting the planted bugs fixed 100%.

Two companion floor checks run too: Brake, refusing a destructive command when the deciding fact is buried in tool output, and Receipt, reading a requirement out of a long log. Every model scores 100 on both so far, so they do not separate models and are not ranked.

The validity gate

Before any model ran, we fixed the checks a run has to pass to count as a measurement of anything: among them that the instrument scores the reference scripts right, that the models separate, that the leader leaves headroom, that the ranking holds when the items are split in half, that longer replies do not score better for being long, and that a rerun on a fresh item set ranks the models alike. A check this run cannot answer yet reads Open, never Passed. A run that fails a check, or leaves one open, is shown as provisional. Of the 7 checks, this run fails 2 (stability and reproducibility), so its ranking is provisional.

CheckResultThis run
InstrumentPassedno-op 0 and revert 100 on every item (checked at admission)
No cheap twinPassedthe only script with no reading of the code is the no-op patch, which scores 0
SeparationPassedspread 14 against the leader's half-width 9
HeadroomPassedleader Gemini 4 Neon at 84
StabilityFailedmean split-half Spearman 0.54 over 200 splits
LengthPassedreply length against score, centred within item: rho 0.13
ReproducibilityFailedlane order Spearman 0.53 against a rerun on a fresh set (5 shared lanes)

The Gauntlet Judgment ranking

Each row is one model, best first: its Gauntlet Judgment score, its 95% interval, its right-file share, and its gap to the leader with that gap’s interval. The grey rows are no edit and bugs reverted. Gauntlet Judgment is not part of Gauntlet Composite.

  1. Bugs revertedreferenceBugs reverted, a reference script checked on every item: Gauntlet Judgment score 100Every item
  2. 1Gemini 4 NeonRight file 92%Gemini 4 Neon: Gauntlet Judgment score 84, 95% interval 75 to 93, right file 92%, the leader95% 75–93Leader
  3. 2GPT-6.1 SolRight file 95%GPT-6.1 Sol: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 95%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
  4. 3Claude Opus 5.5Right file 94%Claude Opus 5.5: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 94%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
  5. 4Claude Fable 5.2Right file 95%Claude Fable 5.2: Gauntlet Judgment score 81, 95% interval 71 to 91, right file 95%, −3.2 points from the leader, 95% interval −12.0 to +5.7, not separated95% 71–91−3.295% −12.0 to +5.7Not separated
  6. 5Claude Sonnet 5.5Right file 94%Claude Sonnet 5.5: Gauntlet Judgment score 75, 95% interval 64 to 85, right file 94%, −9.5 points from the leader, 95% interval −18.1 to −1.0, separated95% 64–85−9.595% −18.1 to −1.0● Separated
  7. 6Claude Fable 5.1Right file 89%Claude Fable 5.1: Gauntlet Judgment score 73, 95% interval 62 to 84, right file 89%, −11.1 points from the leader, 95% interval −21.1 to −1.1, separated95% 62–84−11.195% −21.1 to −1.1● Separated
  8. 7Gemini 4 ArgonRight file 95%Gemini 4 Argon: Gauntlet Judgment score 71, 95% interval 60 to 83, right file 95%, −12.7 points from the leader, 95% interval −22.1 to −3.3, separated95% 60–83−12.795% −22.1 to −3.3● Separated
  9. 8Claude Haiku 5.5Right file 97%Claude Haiku 5.5: Gauntlet Judgment score 70, 95% interval 58 to 81, right file 97%, −14.3 points from the leader, 95% interval −25.0 to −3.6, separated95% 58–81−14.395% −25.0 to −3.6● Separated
  10. No editreferenceNo edit, a reference script checked on every item: Gauntlet Judgment score 0Every item
ModelFixed95% intervalRight file
Gemini 4 Neon84%75–9392%
GPT-6.1 Sol81%71–9195%
Claude Opus 5.581%71–9194%
Claude Fable 5.281%71–9195%
Claude Sonnet 5.575%64–8594%
Claude Fable 5.173%62–8489%
Gemini 4 Argon71%60–8395%
Claude Haiku 5.570%58–8197%

Gauntlet Judgment’s scores come from protocol gauntlet-judgment-patch-v2, item set v11, run e73a52c0, made on . Scores from different protocols or item sets are never compared. The items are private code, so this page shows counts and scores only: never an item’s module, its code, the planted bugs, the failing output or a model’s reply.

What is public and what is private

Public: this method, and for Gauntlet Science, each model’s score with its interval, in-range score, coverage and falsifier rate, each gap with its interval, the floor and ceiling rows, and the gate’s checks. For Gauntlet Ship, the item and component counts, the step budget, each model’s score with its interval, hidden-check, false-done and right-files shares, steps and weakened safeguards, each gap with its interval, the no-change and accepted-fix rows, and the gate’s checks. For Gauntlet Teach, the world count and word limit, both students, each model’s score with its interval, its own Gauntlet Science score on the same worlds, the check student’s score and its mean explanation length, each gap with its interval, the no-explanation and true-law rows, and the gate’s checks. For Gauntlet Judgment, the item count and how many files a model may open, each model’s score with its interval and its right-file share, each gap with its interval, the no-edit and bugs-reverted rows, and the gate’s checks.

Private: the models’ answers, so they cannot be copied into the next round of training data or used to tune a model to the benchmark. Gauntlet Science keeps each world’s law, settings and prompt private too, along with every answer. Gauntlet Ship’s items are private work: every conversation, repository, path, change, hidden check and transcript stays private, and only counts and scores are published. Gauntlet Teach keeps every world, every explanation and every formula a student wrote private. Gauntlet Judgment’s items are private code: every module, its code, the planted bugs, the failing test output and every model’s reply stay private, and only counts and scores are published.

Other benchmarks

The Other benchmarks tab adds outside benchmarks the field respects that measure something neither Artificial Analysis nor the Gauntlet does: what people prefer, research-grade mathematics, a business run for a whole simulated year, and the apps a model works in. Each comes from its maker’s open data file under a license that allows showing it, each view says why it is here and links its source, and each model is named the way the rest of the site names it. Sources whose terms need permission are not shown until it is given.

LMArena

LMArena measures which answers people prefer. Anyone can put one question to two anonymous models and vote for the better reply, and millions of those votes become one rating per model. Artificial Analysis and the Gauntlet score answers by code against known answers; neither asks people which answer they would rather read.

The score is LMArena’s Arena score on its overall text leaderboard, a rating fitted to every head-to-head vote, with the interval LMArena publishes beside it. A higher score means people picked that model’s replies more often. The LMArena view ranks every entry the site can place on one of its models, under the site’s own names, and lists the rest under LMArena’s own names. 42 of LMArena’s 421 entries are models this site shows, ranked above under the site’s names. The other 379 are listed under LMArena’s own names.

Data: LMArena’s text leaderboard, from its open dataset, under CC BY 4.0, data of . Synced from the open data file, never from the page.

FrontierMath

FrontierMath measures hard mathematics. Its problems were written by mathematicians, run from difficult undergraduate problems to ones an advanced graduate student would find exploratory, and have never been published, so no model can have read them; each answer is checked by computer. Neither Artificial Analysis’s index nor the Gauntlet sets research-grade mathematics.

The score is the share of tiers 1 to 3 that a model at that setting solved in Epoch AI’s runs, with a 95% interval from Epoch AI’s standard error. The FrontierMath view ranks every entry the site can place on one of its models, under the site’s own names, and lists the rest under FrontierMath’s own names. 38 of FrontierMath’s 125 entries are models this site shows, ranked above under the site’s names. The other 87 are listed under FrontierMath’s own names.

Data: FrontierMath by Epoch AI, from Epoch AI’s open benchmark data, under CC BY 4.0, data of . Synced from the open data file, never from the page.

Vending-Bench 2

Vending-Bench 2 measures whether a model stays on course over a very long task. The model runs a simulated vending-machine business for a simulated year from a $500 start, finding suppliers, ordering stock, setting prices and handling refunds, and is scored on the money in its bank account at the end. Neither Artificial Analysis nor the Gauntlet runs one task anywhere near that long.

The score is the bank balance in dollars after the simulated year, averaged over runs, as Epoch AI publishes Andon Labs’ results. A model that loses money ends below its $500 start. The Vending-Bench 2 view ranks every entry the site can place on one of its models, under the site’s own names, and lists the rest under Vending-Bench 2’s own names. 21 of Vending-Bench 2’s 72 entries are models this site shows, ranked above under the site’s names. The other 51 are listed under Vending-Bench 2’s own names.

Data: Vending-Bench 2 by Andon Labs, results from Epoch AI’s open benchmark data, under CC BY 4.0, data of . Synced from the open data file, never from the page.

ADEMark

ADEMark measures the app around the model, not the model. Every app here runs the same model, Claude Opus 5.5, effort high, on one Mac mini M4, 16 GB, macOS 27, and is timed on how fast it starts, how far typing lags while five agents stream, how much memory it holds and how many tokens it sends for a plain “hi”. Artificial Analysis and the Gauntlet measure models; neither measures the harness a model works in, which is what decides how an agent feels to use. Because it ranks apps, it is in neither External Composite nor Everything Composite.

Each figure is the median of an app’s valid runs. Lower is better on all four, and each test is ranked on its own, so there is no overall winner: First marks the best app on each test. The Bare CLI, Claude Code alone in a terminal, is ADEMark’s control and is never ranked. Did not finish and Not measured are ADEMark’s own words for a gap; its site gives the cause of each. Its view is a table of apps, never a ranking of models.

Its view. Data: ADEMark’s published results, run control-2026-11-10T08-31-19Z, under CC BY 4.0, data of .

External Composite

External Composite puts the outside benchmarks together, and leaves out Artificial Analysis and the Gauntlet. Each model’s place in each source is its percentile among every model that source ranks, 100 for the source’s top model and 0 for its bottom, so an arena rating, a share of math problems solved and a dollar balance count alike. The dot is the mean of those percentiles, and the open marks are the parts. Only the 29 models in at least two of LMArena, FrontierMath and Vending-Bench 2 are ranked; ADEMark ranks apps, not models, so it is not part of it.

Its view is the tab’s first.

Everything Composite

Everything Composite puts the outside benchmarks and our own together. Each model’s dot is the mean of two percentiles: where its External Composite stands among every External Composite model, and where its Gauntlet Composite stands among every Gauntlet Composite model. Both parts are marked and named on its row. Only the 7 models with both are ranked. Gauntlet Composite here is the same ranking as the UL Gauntlet tab’s, the mean of Gauntlet Science, Gauntlet Ship and Gauntlet Teach.

Its view shows both parts on every row.

Who made this

AI Benchmarks was created by Daniel Miessler. It is independent, and not affiliated with Artificial Analysis, or with Theo or t3.gg. Gauntlet Science, Gauntlet Ship, Gauntlet Teach and Gauntlet Judgment are Unsupervised Learning’s own.