ai benchmarks

Real-world AI performance metrics

Gauntlet Composite updated

Gauntlet Science updated

Gauntlet Ship updated

Gauntlet Teach updated

Gauntlet Judgment updated

Artificial Analysis data updated

Every change Artificial Analysis made to its scores and prices, newest first. Click a model to see its past scores.

12 price changes

Google model costs increased across all six listed releases, including Gemini 3.5 Flash at high effort from $1.563 to $3.694 per Intelligence task and Gemini 3.1 Pro Preview from $0.6747 to $1.299. Grok 4.6 costs fell at every listed effort level, from $0.3758 at low effort to $1.752 at xhigh effort, while Grok 4.5 high fell from $1.036 to $0.8807. DeepSeek V3 0324 rose slightly from $0.0202 to $0.0211.

AI-written summary of these changes · gpt-5.6-terra

Price changes 12

  • highCost per task$1.56 → $3.69+$2.13
  • single settingCost per task$0.67 → $1.30+$0.62
  • highCost per task$0.93 → $1.60+$0.67
  • single settingCost per task$0.12 → $0.19+$0.069
  • single settingCost per task$0.039 → $0.060+$0.021
  • single settingCost per task$0.23 → $0.33+$0.099
  • lowCost per task$0.48 → $0.38−$0.099
  • Grok 4.6mediumCost per task$1.50 → $1.14−$0.35
  • Grok 4.6highCost per task$1.86 → $1.48−$0.37
  • Grok 4.6xhighCost per task$2.32 → $1.75−$0.57
  • highCost per task$1.04 → $0.88−$0.16
  • single settingCost per task$0.020 → $0.021+$0.0009

1 new release

InclusionAI's Ling 3.1 Flash entered the Intelligence Index at 41.1, with one listed setting costing $0.99 per task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 41.1one setting, $0.99 per task

2 price changes

MiMo-V2.6-Flash's default cost per Intelligence Index task rose from $0.0622 to $0.0623. GPT-6 Luna's max-effort cost rose from $0.0677 to $0.0678. Both changes were increases of $0.0001 per task.

AI-written summary of these changes · gpt-5.6-terra

Price changes 2

  • single settingCost per task$0.062 → $0.062+$0.0001
  • maxCost per task$0.068 → $0.068+$0.0001

2 score changes · 4 price changes · 1 new effort level

Claude Sonnet 5.5 added a low reasoning-effort setting scoring 35.9 on Intelligence at $0.42 per task. Its medium and high Intelligence scores each rose by 0.1 points, to 40.8 and 46.8. Costs increased slightly at medium ($0.5862 to $0.5894), high ($1.08 to $1.123), xhigh ($2.743 to $2.746), and max ($7.621 to $7.667).

AI-written summary of these changes · gpt-5.6-terra

Score changes 2

  • mediumIntelligence40.7 → 40.8+0.1
  • Claude Sonnet 5.5highIntelligence46.7 → 46.8+0.1

Price changes 4

  • mediumCost per task$0.59 → $0.59+$0.0032
  • Claude Sonnet 5.5highCost per task$1.08 → $1.12+$0.043
  • Claude Sonnet 5.5xhighCost per task$2.74 → $2.75+$0.0036
  • Claude Sonnet 5.5maxCost per task$7.62 → $7.67+$0.046

New effort levels 1

  • lowIntelligence35.9$0.42 per task

1 new effort level

Grok 4.7 added a low reasoning-effort result of 42.2 on the Intelligence Index, with a cost of $1.25 per task.

AI-written summary of these changes · gpt-5.6-terra

New effort levels 1

  • lowIntelligence42.2$1.25 per task

9 score changes · 8 price changes

GPT-6 Luna’s Intelligence score rose at every effort level, from 20.9 to 21.5 at low through 37.3 to 38.1 at max. GPT-6 Sol’s scores moved from 33.9 to 34.2 at low, 44.1 to 44.2 at xhigh and 47.5 to 47.6 at max, while high fell from 42.8 to 42.4. Both models had small mixed cost movements: Luna’s xhigh cost rose from $0.0421 to $0.0422, while Sol’s low cost rose from $0.1326 to $0.1328 and its other listed costs fell.

AI-written summary of these changes · gpt-5.6-terra

Score changes 9

  • lowIntelligence20.9 → 21.5+0.6
  • GPT-6 LunamediumIntelligence29.5 → 29.9+0.4
  • GPT-6 LunahighIntelligence32.1 → 32.9+0.8
  • GPT-6 LunaxhighIntelligence33.9 → 34.6+0.7
  • GPT-6 LunamaxIntelligence37.3 → 38.1+0.8
  • lowIntelligence33.9 → 34.2+0.3
  • GPT-6 SolhighIntelligence42.8 → 42.4−0.4
  • GPT-6 SolxhighIntelligence44.1 → 44.2+0.1
  • GPT-6 SolmaxIntelligence47.5 → 47.6+0.1

Price changes 8

  • lowCost per task$0.13 → $0.13+$0.0002
  • GPT-6 SolhighCost per task$0.38 → $0.37−$0.0025
  • GPT-6 SolxhighCost per task$0.52 → $0.52−$0.0028
  • GPT-6 SolmaxCost per task$1.05 → $1.04−$0.0047
  • mediumCost per task$0.018 → $0.018−$0.0001
  • GPT-6 LunahighCost per task$0.029 → $0.029−$0.0001
  • GPT-6 LunaxhighCost per task$0.042 → $0.042+$0.0001
  • GPT-6 LunamaxCost per task$0.068 → $0.068−$0.0002

1 new release

Upstage’s new Solar Mini 4 release scores 24.1 on the Intelligence Index. Its single listed setting costs $0.36 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 24.1one setting, $0.36 per task

1 price change

NVIDIA's Nemotron 3.5 Lightning default cost per Intelligence Index task fell from $0.1004 to $0.0931, a decrease of $0.0073 (about 7%).

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.10 → $0.093−$0.0073

1 new release

Google’s new Gemini 4 Argon release scores 52.6 on the Intelligence Index in its listed setting. Its cost is $1.99 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 52.6one setting, $1.99 per task

1 price change

Claude Sonnet 5.5 at max reasoning effort rose in cost from $7.603 to $7.621 per Intelligence Index task. This was an increase of $0.018, with no score, release, or other effort-level changes listed.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • maxCost per task$7.60 → $7.62+$0.018

1 new release · 2 new effort levels

Kimi K3 added a low-effort setting scoring 30.1 on Intelligence at $1.15 per task. Inkling Small debuted at 25.7 for $0.089 per task, while DeepSeek V4 Flash 0420's new high-effort setting scored 24.4 at $0.13. The two lower-scoring entries are also far cheaper per task than Kimi K3's low-effort setting.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 25.7one setting, $0.089 per task

New effort levels 2

  • lowIntelligence30.1$1.15 per task
  • highIntelligence24.4$0.13 per task

1 removed effort level

Claude Sonnet 5.5’s low reasoning-effort option was removed from the Intelligence Index. It had scored 35.8 and cost $0.41 per task.

AI-written summary of these changes · gpt-5.6-terra

Removed effort levels 1

  • lowIntelligencewas 35.8was $0.41 per task

1 new release

OpenAI's new GPT-6.1 Sol entered the Intelligence Index at 51.8. It has five effort levels, with Intelligence Index task costs ranging from $0.13 to $0.72.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 51.85 effort levels, $0.13–$0.72 per task

1 price change

DeepSeek V3 0324’s cost per Intelligence Index task fell from $0.1053 to $0.0202, a reduction of $0.0851.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.11 → $0.020−$0.085

7 price changes

GPT-6 Luna’s cost per Intelligence task rose at medium, high, and xhigh effort, from $0.0173 to $0.0176, $0.0286 to $0.0291, and $0.0417 to $0.0421, while max fell slightly from $0.0681 to $0.0679. GPT-6 Sol’s xhigh and max costs fell from $0.5319 to $0.5236 and $1.056 to $1.049, while high increased from $0.3746 to $0.3753.

AI-written summary of these changes · gpt-5.6-terra

Price changes 7

  • mediumCost per task$0.017 → $0.018+$0.0003
  • GPT-6 LunahighCost per task$0.029 → $0.029+$0.0005
  • GPT-6 LunaxhighCost per task$0.042 → $0.042+$0.0004
  • GPT-6 LunamaxCost per task$0.068 → $0.068−$0.0002
  • highCost per task$0.37 → $0.38+$0.0007
  • GPT-6 SolxhighCost per task$0.53 → $0.52−$0.0083
  • GPT-6 SolmaxCost per task$1.06 → $1.05−$0.0070

2 price changes

GPT-6 Sol’s medium-effort Intelligence task cost edged down from $0.2482 to $0.2472. Its low-effort cost moved slightly up from $0.1322 to $0.1326, leaving the two effort levels moving in opposite directions.

AI-written summary of these changes · gpt-5.6-terra

Price changes 2

  • lowCost per task$0.13 → $0.13+$0.0004
  • GPT-6 SolmediumCost per task$0.25 → $0.25−$0.0010

1 new release

Alibaba’s new Qwen3.5 9B release scores 11.2 on the Intelligence Index in its listed setting. Its cost is $0.21 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 11.2one setting, $0.21 per task

1 price change

Tencent's Hy3 default cost per Intelligence Index task fell from $0.0743 to $0.0718, a decrease of $0.0025.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.074 → $0.072−$0.0025

1 new release

Anthropic's new Claude Sonnet 5.5 release scores 56 on the Intelligence Index. It has five reasoning-effort levels, with Intelligence Index task costs ranging from $0.41 to $7.60.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 56.05 effort levels, $0.41–$7.60 per task

1 new release

Xiaomi’s MiMo-V2.6-Flash was added with an Intelligence Index score of 37.9. Its single listed setting costs $0.062 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 37.9one setting, $0.062 per task
Load older