ai benchmarks

Real-world AI performance metrics

Gauntlet Composite updated

Gauntlet Science updated

Gauntlet Ship updated

Gauntlet Teach updated

Gauntlet Judgment updated

Artificial Analysis data updated

Every change Artificial Analysis made to its scores and prices, newest first. Click a model to see its past scores.

1 price change

Price changes 1

  • single settingCost per task$0.56 → $0.54−$0.018

1 new release

New releases 1

  • peak 27.8one setting, $0.58 per task

1 new release · 2 new effort levels

New releases 1

  • peak 17.4one setting, $2.37 per task

New effort levels 2

  • lowIntelligence15.7$1.91 per task
  • Qwen3.8 27BmediumIntelligence17.5$2.16 per task

2 new releases · 1 price change

New releases 2

  • peak 5.5one setting, $0.29 per task
  • peak 4.3one setting, $0.040 per task

Price changes 1

  • single settingCost per task$5.54 → $4.21−$1.33

1 new release

New releases 1

  • peak 26.02 effort levels, $1.41–$2.06 per task

5 price changes

Price changes 5

  • lowCost per task$4.49 → $4.47−$0.016
  • Claude Fable 5.1mediumCost per task$5.58 → $5.56−$0.016
  • Claude Fable 5.1highCost per task$7.56 → $7.54−$0.021
  • Claude Fable 5.1xhighCost per task$10.89 → $10.85−$0.038
  • Claude Fable 5.1maxCost per task$13.04 → $12.99−$0.045

1 new release · 88 removed releases · 115 score changes · 70 price changes · 3 removed effort levels · 1 index version change

Index version 1

  • Intelligence Indexv4.3 → v5

New releases 1

  • peak 59.4one setting, $7.76 per task

Score changes 115

  • lowIntelligence45.8 → 43.0−2.8
  • GPT-6 AstralowCoding75.7 → —dropped
  • GPT-6 AstralowAgentic38.8 → —dropped
  • GPT-6 AstramediumIntelligence49.6 → 51.5+1.9
  • GPT-6 AstramediumCoding76.7 → —dropped
  • GPT-6 AstramediumAgentic46.0 → —dropped
  • GPT-6 AstrahighIntelligence50.9 → 54.8+3.9
  • GPT-6 AstrahighCoding77.1 → —dropped
  • GPT-6 AstrahighAgentic48.2 → —dropped
  • GPT-6 AstraxhighIntelligence52.4 → 55.3+2.9
  • GPT-6 AstraxhighCoding75.9 → —dropped
  • GPT-6 AstraxhighAgentic50.2 → —dropped
Show 103 more
  • maxIntelligence52.7 → 56.9+4.2
  • GPT-6 AstramaxCoding76.9 → —dropped
  • GPT-6 AstramaxAgentic51.0 → —dropped
  • lowIntelligence46.8 → 38.5−8.3
  • Claude Fable 5.1lowCoding75.2 → —dropped
  • Claude Fable 5.1lowAgentic47.1 → —dropped
  • Claude Fable 5.1mediumIntelligence48.9 → 43.0−5.9
  • Claude Fable 5.1mediumCoding77.1 → —dropped
  • Claude Fable 5.1mediumAgentic50.2 → —dropped
  • Claude Fable 5.1highIntelligence51.2 → 47.0−4.2
  • Claude Fable 5.1highCoding79.1 → —dropped
  • Claude Fable 5.1highAgentic53.1 → —dropped
  • Claude Fable 5.1xhighIntelligence53.2 → 49.7−3.5
  • Claude Fable 5.1xhighCoding80.7 → —dropped
  • Claude Fable 5.1xhighAgentic57.2 → —dropped
  • Claude Fable 5.1maxIntelligence53.4 → 49.0−4.4
  • Claude Fable 5.1maxCoding81.6 → —dropped
  • Claude Fable 5.1maxAgentic57.9 → —dropped
  • maxIntelligence48.1 → 36.4−11.7
  • Muse Spark 1.3maxCoding75.8 → —dropped
  • Muse Spark 1.3maxAgentic55.5 → —dropped
  • single settingIntelligence45.4 → 33.6−11.8
  • Qwen3.8 Max (0902)single settingCoding76.2 → —dropped
  • Qwen3.8 Max (0902)single settingAgentic56.0 → —dropped
  • lowIntelligence34.3 → 23.1−11.2
  • GLM-5.3maxIntelligence44.8 → 31.7−13.1
  • GLM-5.3maxCoding74.8 → —dropped
  • GLM-5.3maxAgentic53.1 → —dropped
  • mediumIntelligence39.8 → 30.0−9.8
  • Gemini 3.8 FlashmediumCoding74.1 → —dropped
  • Gemini 3.8 FlashmediumAgentic39.7 → —dropped
  • Gemini 3.8 FlashhighIntelligence40.9 → 30.5−10.4
  • Gemini 3.8 FlashhighCoding76.3 → —dropped
  • Gemini 3.8 FlashhighAgentic40.2 → —dropped
  • maxIntelligence36.0 → 23.1−12.9
  • DeepSeek V4 Pro 0813maxCoding68.8 → —dropped
  • DeepSeek V4 Pro 0813maxAgentic41.3 → —dropped
  • xhighIntelligence33.7 → 20.1−13.6
  • Qwen3.8 27BxhighCoding68.1 → —dropped
  • Qwen3.8 27BxhighAgentic45.8 → —dropped
  • single settingIntelligence29.2 → 18.7−10.5
  • MiniMax-M3single settingCoding58.6 → —dropped
  • MiniMax-M3single settingAgentic29.5 → —dropped
  • single settingIntelligence25.7 → 12.9−12.8
  • Inkling Smallsingle settingCoding52.9 → —dropped
  • Inkling Smallsingle settingAgentic23.5 → —dropped
  • xhighIntelligence25.0 → 12.5−12.5
  • InklingxhighCoding52.1 → —dropped
  • InklingxhighAgentic22.5 → —dropped
  • single settingIntelligence22.9 → 11.8−11.1
  • Nemotron 3 Ultra 550B A55Bsingle settingCoding49.3 → —dropped
  • Nemotron 3 Ultra 550B A55Bsingle settingAgentic20.1 → —dropped
  • single settingIntelligence22.2 → 11.7−10.5
  • Gemini 3.5 Flash-Litesingle settingCoding49.3 → —dropped
  • Gemini 3.5 Flash-Litesingle settingAgentic14.3 → —dropped
  • highIntelligence17.5 → 6.3−11.2
  • Muse GlimmerhighCoding49.0 → —dropped
  • Muse GlimmerhighAgentic8.5 → —dropped
  • single settingIntelligence46.3 → 29.9−16.4
  • single settingIntelligence43.7 → 28.5−15.2
  • single settingIntelligence43.9 → 29.5−14.4
  • xhighIntelligence38.9 → 24.7−14.2
  • single settingIntelligence42.7 → 28.9−13.8
  • single settingIntelligence38.4 → 25.3−13.1
  • single settingIntelligence26.5 → 13.6−12.9
  • maxIntelligence45.1 → 32.5−12.6
  • single settingIntelligence30.3 → 17.7−12.6
  • lowIntelligence42.2 → 30.9−11.3
  • Grok 4.7highIntelligence46.3 → 35.0−11.3
  • Grok 4.7xhighIntelligence46.4 → 34.3−12.1
  • lowIntelligence21.5 → 9.7−11.8
  • GPT-6 LunamediumIntelligence29.9 → 18.3−11.6
  • GPT-6 LunahighIntelligence32.9 → 21.4−11.5
  • GPT-6 LunaxhighIntelligence34.6 → 23.3−11.3
  • GPT-6 LunamaxIntelligence38.1 → 27.8−10.3
  • single settingIntelligence37.9 → 26.7−11.2
  • lowIntelligence29.5 → 18.6−10.9
  • Claude Haiku 5.5mediumIntelligence34.5 → 24.6−9.9
  • Claude Haiku 5.5highIntelligence37.8 → 29.6−8.2
  • Claude Haiku 5.5xhighIntelligence41.2 → 34.3−6.9
  • Claude Haiku 5.5maxIntelligence43.4 → 36.2−7.2
  • lowIntelligence35.9 → 25.3−10.6
  • Claude Sonnet 5.5mediumIntelligence40.8 → 30.3−10.5
  • Claude Sonnet 5.5highIntelligence46.8 → 41.1−5.7
  • Claude Sonnet 5.5xhighIntelligence51.9 → 49.6−2.3
  • Claude Sonnet 5.5maxIntelligence56.0 → 54.2−1.8
  • single settingIntelligence55.1 → 46.0−9.1
  • maxIntelligence52.4 → 43.3−9.1
  • lowIntelligence42.3 → 34.5−7.8
  • Claude Opus 5.5mediumIntelligence51.2 → 48.1−3.1
  • Claude Opus 5.5highIntelligence53.6 → 51.7−1.9
  • Claude Opus 5.5xhighIntelligence56.0 → 57.0+1.0
  • Claude Opus 5.5maxIntelligence57.6 → 58.4+0.8
  • lowIntelligence42.1 → 34.5−7.6
  • GPT-6.1 SolmediumIntelligence47.8 → 46.1−1.7
  • GPT-6.1 SolxhighIntelligence51.0 → 52.3+1.3
  • GPT-6.1 SolmaxIntelligence51.8 → 54.5+2.7
  • lowIntelligence36.2 → 29.2−7.0
  • GPT-6.1 TerramediumIntelligence41.3 → 38.1−3.2
  • GPT-6.1 TerrahighIntelligence43.9 → 42.3−1.6
  • GPT-6.1 TerraxhighIntelligence45.4 → 45.0−0.4
  • GPT-6.1 TerramaxIntelligence48.1 → 51.0+2.9
  • highIntelligence52.6 → 50.4−2.2

Price changes 70

  • single settingCost per task$0.37 → $2.23+$1.86
  • highCost per task$1.62 → $7.68+$6.06
  • single settingCost per task$0.17 → $0.69+$0.52
  • single settingCost per task$1.61 → $5.54+$3.93
  • xhighCost per task$1.15 → $3.53+$2.38
  • single settingCost per task$0.19 → $0.56+$0.37
  • lowCost per task$1.03 → $2.92+$1.89
  • Grok 4.7highCost per task$2.36 → $6.18+$3.82
  • Grok 4.7xhighCost per task$3.24 → $7.51+$4.28
  • highCost per task$0.058 → $0.16+$0.10
  • maxCost per task$0.67 → $1.77+$1.10
  • single settingCost per task$0.51 → $1.29+$0.78
Show 58 more
  • lowCost per task$0.0045 → $0.0051+$0.0006
  • GPT-6 LunamediumCost per task$0.018 → $0.042+$0.025
  • GPT-6 LunahighCost per task$0.029 → $0.073+$0.044
  • GPT-6 LunaxhighCost per task$0.042 → $0.10+$0.062
  • GPT-6 LunamaxCost per task$0.068 → $0.15+$0.084
  • lowCost per task$0.85 → $1.57+$0.72
  • GLM-5.3maxCost per task$2.01 → $5.01+$3.00
  • mediumCost per task$2.59 → $6.26+$3.67
  • Gemini 3.8 FlashhighCost per task$3.48 → $7.77+$4.28
  • xhighCost per task$0.84 → $2.02+$1.17
  • lowCost per task$0.055 → $0.071+$0.017
  • GPT-6.1 TerramediumCost per task$0.095 → $0.17+$0.074
  • GPT-6.1 TerrahighCost per task$0.13 → $0.26+$0.12
  • GPT-6.1 TerraxhighCost per task$0.19 → $0.40+$0.21
  • GPT-6.1 TerramaxCost per task$0.40 → $0.96+$0.56
  • single settingCost per task$0.12 → $0.28+$0.16
  • maxCost per task$0.26 → $0.60+$0.34
  • single settingCost per task$1.03 → $2.35+$1.32
  • xhighCost per task$0.60 → $1.35+$0.76
  • single settingCost per task$0.089 → $0.20+$0.11
  • single settingCost per task$0.22 → $0.49+$0.27
  • single settingCost per task$1.49 → $3.19+$1.70
  • maxCost per task$3.27 → $6.95+$3.68
  • single settingCost per task$0.062 → $0.13+$0.066
  • lowCost per task$0.35 → $0.51+$0.17
  • Claude Sonnet 5.5mediumCost per task$0.48 → $0.78+$0.30
  • Claude Sonnet 5.5highCost per task$0.88 → $1.64+$0.75
  • Claude Sonnet 5.5xhighCost per task$2.01 → $4.09+$2.08
  • Claude Sonnet 5.5maxCost per task$5.46 → $10.02+$4.56
  • lowCost per task$0.025 → $0.044+$0.019
  • Claude Haiku 5.5mediumCost per task$0.047 → $0.092+$0.045
  • Claude Haiku 5.5highCost per task$0.079 → $0.16+$0.079
  • Claude Haiku 5.5xhighCost per task$0.12 → $0.25+$0.12
  • Claude Haiku 5.5maxCost per task$0.21 → $0.41+$0.20
  • lowCost per task$0.13 → $0.16+$0.031
  • GPT-6.1 SolmediumCost per task$0.21 → $0.36+$0.15
  • GPT-6.1 SolhighCost per task$0.32 → $0.59+$0.27
  • GPT-6.1 SolxhighCost per task$0.39 → $0.76+$0.37
  • GPT-6.1 SolmaxCost per task$0.72 → $1.27+$0.55
  • lowCost per task$2.37 → $4.49+$2.12
  • Claude Fable 5.1mediumCost per task$2.98 → $5.58+$2.60
  • Claude Fable 5.1highCost per task$3.91 → $7.56+$3.65
  • Claude Fable 5.1xhighCost per task$5.98 → $10.89+$4.91
  • Claude Fable 5.1maxCost per task$7.63 → $13.04+$5.41
  • single settingCost per task$0.13 → $0.26+$0.12
  • lowCost per task$0.82 → $1.42+$0.61
  • GPT-6 AstramediumCost per task$1.54 → $2.73+$1.19
  • GPT-6 AstrahighCost per task$1.73 → $3.24+$1.51
  • GPT-6 AstraxhighCost per task$2.31 → $3.66+$1.35
  • GPT-6 AstramaxCost per task$3.26 → $6.24+$2.99
  • lowCost per task$0.55 → $0.98+$0.42
  • Claude Opus 5.5mediumCost per task$1.34 → $2.45+$1.12
  • Claude Opus 5.5highCost per task$1.82 → $3.46+$1.64
  • Claude Opus 5.5xhighCost per task$3.46 → $6.05+$2.59
  • Claude Opus 5.5maxCost per task$5.98 → $10.04+$4.05
  • single settingCost per task$5.41 → $10.02+$4.62
  • maxCost per task$1.31 → $2.39+$1.07
  • single settingCost per task$2.11 → $3.68+$1.57

Removed effort levels 3

  • xhighIntelligencewas 45.1was $1.12 per task
  • lowIntelligencewas 26.2was $1.17 per task
  • Qwen3.8 27BmediumIntelligencewas 27.6was $1.27 per task

Removed releases 88

  • was 50.8 peakhad 5 effort levels
  • was 49.6 peakhad one setting
  • was 47.6 peakhad 5 effort levels
  • was 47.0 peakhad 5 effort levels
  • was 44.3 peakhad 4 effort levels
  • was 43.6 peakhad 2 effort levels
  • was 42.1 peakhad 5 effort levels
  • was 41.8 peakhad one setting
  • was 41.8 peakhad one setting
  • was 41.1 peakhad one setting
  • was 40.2 peakhad one setting
  • was 39.9 peakhad one setting
Show 76 more
  • was 39.8 peakhad one setting
  • was 39.6 peakhad one setting
  • was 39.5 peakhad one setting
  • was 38.8 peakhad one setting
  • was 38.4 peakhad 3 effort levels
  • was 38.2 peakhad 5 effort levels
  • was 37.3 peakhad 5 effort levels
  • was 34.8 peakhad one setting
  • was 34.3 peakhad one setting
  • was 34.0 peakhad one setting
  • was 33.7 peakhad one setting
  • was 33.7 peakhad one setting
  • was 32.6 peakhad one setting
  • was 30.4 peakhad one setting
  • was 30.1 peakhad one setting
  • was 29.7 peakhad one setting
  • was 29.5 peakhad one setting
  • was 27.0 peakhad one setting
  • was 26.7 peakhad one setting
  • was 26.6 peakhad one setting
  • was 26.4 peakhad one setting
  • was 26.1 peakhad one setting
  • was 26.0 peakhad one setting
  • was 26.0 peakhad one setting
  • was 25.8 peakhad one setting
  • was 25.3 peakhad one setting
  • was 25.2 peakhad one setting
  • was 24.9 peakhad one setting
  • was 24.4 peakhad 2 effort levels
  • was 24.1 peakhad one setting
  • was 24.1 peakhad one setting
  • was 22.8 peakhad one setting
  • was 21.8 peakhad one setting
  • was 21.4 peakhad one setting
  • was 20.7 peakhad one setting
  • was 20.7 peakhad one setting
  • was 20.1 peakhad one setting
  • was 19.1 peakhad one setting
  • was 18.4 peakhad one setting
  • was 18.2 peakhad one setting
  • was 16.9 peakhad one setting
  • was 16.8 peakhad one setting
  • was 16.6 peakhad one setting
  • was 16.1 peakhad one setting
  • was 15.6 peakhad one setting
  • was 15.6 peakhad one setting
  • was 14.2 peakhad one setting
  • was 12.9 peakhad one setting
  • was 12.8 peakhad one setting
  • was 12.7 peakhad one setting
  • was 12.3 peakhad one setting
  • was 11.6 peakhad one setting
  • was 11.4 peakhad one setting
  • was 11.3 peakhad one setting
  • was 11.2 peakhad one setting
  • was 11.1 peakhad one setting
  • was 11.0 peakhad one setting
  • was 10.8 peakhad one setting
  • was 10.8 peakhad one setting
  • was 9.7 peakhad one setting
  • was 9.3 peakhad one setting
  • was 9.3 peakhad one setting
  • was 9.2 peakhad one setting
  • was 9.1 peakhad one setting
  • was 9.0 peakhad one setting
  • was 8.9 peakhad one setting
  • was 8.9 peakhad one setting
  • was 8.5 peakhad one setting
  • was 7.8 peakhad one setting
  • was 7.1 peakhad one setting
  • was 7.0 peakhad one setting
  • was 6.5 peakhad one setting
  • was 6.3 peakhad one setting
  • was 6.0 peakhad one setting
  • was 5.5 peakhad one setting
  • was 4.8 peakhad one setting

1 price change

Price changes 1

  • maxCost per task$0.17 → $0.18+$0.0031

1 price change

Price changes 1

  • maxCost per task$0.15 → $0.17+$0.024

1 removed release

Removed releases 1

  • was 12.7 peakhad one setting

2 price changes

Price changes 2

  • xhighCost per task$1.29 → $0.81−$0.48
  • xhighCost per task$1.69 → $1.13−$0.56

1 new release · 4 price changes · 1 new effort level

New releases 1

  • peak 26.4one setting, $0.46 per task

Price changes 4

  • xhighCost per task$2.15 → $1.12−$1.03
  • Muse Spark 1.3maxCost per task$3.69 → $1.31−$2.38
  • maxCost per task$8.63 → $3.27−$5.36
  • single settingCost per task$0.079 → $0.086+$0.0069

New effort levels 1

  • mediumIntelligence39.8$2.59 per task

3 price changes

Price changes 3

  • single settingCost per task$0.14 → $0.12−$0.018
  • highCost per task$0.12 → $0.12+$0.0047
  • maxCost per task$0.32 → $0.32−$0.0016

1 new release

New releases 1

  • peak 48.15 effort levels, $0.055–$0.40 per task

6 new releases · 1 removed release · 48 price changes · 1 removed effort level

New releases 6

  • peak 26.7one setting, $0.15 per task
  • peak 25.0one setting, $0.60 per task
  • peak 22.8one setting, $0.11 per task
  • peak 20.1one setting, $0.028 per task
  • peak 12.7one setting, $0.080 per task
  • peak 11.0one setting, $0.12 per task

Price changes 48

  • highCost per task$1.24 → $3.48+$2.24
  • single settingCost per task$0.039 → $0.089+$0.051
  • xhighCost per task$1.37 → $2.15+$0.79
  • Muse Spark 1.3maxCost per task$1.60 → $3.69+$2.09
  • maxCost per task$4.54 → $8.63+$4.09
  • single settingCost per task$0.23 → $0.14−$0.085
  • maxCost per task$0.48 → $0.32−$0.16
  • single settingCost per task$1.13 → $1.61+$0.47
  • single settingCost per task$0.37 → $0.51+$0.15
  • single settingCost per task$0.079 → $0.058−$0.021
  • single settingCost per task$1.02 → $0.75−$0.27
  • xhighCost per task$0.97 → $1.29+$0.32
Show 36 more
  • single settingCost per task$0.47 → $0.37−$0.10
  • mediumCost per task$0.90 → $0.77−$0.14
  • GPT-5.5highCost per task$1.54 → $1.40−$0.14
  • GPT-5.5xhighCost per task$2.63 → $2.10−$0.53
  • highCost per task$0.10 → $0.13+$0.025
  • DeepSeek V4 Flash 0420maxCost per task$0.089 → $0.098+$0.0088
  • highCost per task$3.69 → $2.98−$0.72
  • single settingCost per task$1.15 → $0.93−$0.22
  • highCost per task$1.99 → $1.62−$0.37
  • xhighCost per task$1.38 → $1.69+$0.31
  • single settingCost per task$0.69 → $0.57−$0.12
  • lowCost per task$1.25 → $1.03−$0.22
  • Grok 4.7highCost per task$2.73 → $2.36−$0.36
  • Grok 4.7xhighCost per task$3.74 → $3.24−$0.50
  • xhighCost per task$0.45 → $0.37−$0.076
  • single settingCost per task$0.28 → $0.23−$0.047
  • single settingCost per task$0.50 → $0.43−$0.073
  • single settingCost per task$0.12 → $0.100−$0.017
  • highCost per task$0.88 → $0.76−$0.12
  • maxCost per task$1.47 → $1.28−$0.19
  • single settingCost per task$0.015 → $0.017+$0.0021
  • lowCost per task$1.05 → $1.17+$0.12
  • Qwen3.8 27BmediumCost per task$1.13 → $1.27+$0.14
  • Qwen3.8 27BxhighCost per task$1.01 → $1.15+$0.14
  • single settingCost per task$0.80 → $0.71−$0.093
  • single settingCost per task$0.19 → $0.17−$0.020
  • single settingCost per task$1.30 → $1.44+$0.14
  • single settingCost per task$0.038 → $0.042+$0.0038
  • single settingCost per task$0.14 → $0.15+$0.0096
  • single settingCost per task$0.33 → $0.31−$0.016
  • single settingCost per task$0.021 → $0.021−$0.0006
  • single settingCost per task$0.19 → $0.19−$0.0031
  • maxCost per task$0.15 → $0.15+$0.0013
  • single settingCost per task$0.072 → $0.072+$0.0006
  • single settingCost per task$0.47 → $0.47+$0.0020
  • single settingCost per task$0.080 → $0.079−$0.0002

Removed effort levels 1

  • mediumIntelligencewas 39.8was $0.93 per task

Removed releases 1

  • was 26.4 peakhad one setting

1 price change

Price changes 1

  • highCost per task$0.014 → $0.011−$0.0031

1 price change

Price changes 1

  • maxCost per task$0.40 → $0.48+$0.080

1 new release

New releases 1

  • peak 55.1one setting, $2.11 per task

1 price change

Price changes 1

  • single settingCost per task$1.00 → $1.01+$0.0088

2 price changes

Price changes 2

  • single settingCost per task$0.054 → $0.14+$0.086
  • single settingCost per task$0.019 → $0.038+$0.019
Load older