ai benchmarks

Real-world AI performance metrics

Gauntlet Composite updated

Gauntlet Science updated

Gauntlet Ship updated

Gauntlet Teach updated

Gauntlet Judgment updated

Artificial Analysis data updated

Every change Artificial Analysis made to its scores and prices, newest first. Click a model to see its past scores.

1 price change

Price changes 1

  • maxCost per task$4.52 → $4.54+$0.024

1 removed release

Removed releases 1

  • was 26.7 peakhad one setting

1 new release

New releases 1

  • peak 26.6one setting, $0.71 per task

1 new release

New releases 1

  • peak 43.9one setting, $1.49 per task

1 removed release · 1 price change

Price changes 1

  • single settingCost per task$1.00 → $1.00+$0.0022

Removed releases 1

  • was 26.6 peakhad one setting

1 price change

Price changes 1

  • single settingCost per task$0.51 → $0.23−$0.29

4 price changes

Price changes 4

  • maxCost per task$0.57 → $0.40−$0.17
  • single settingCost per task$0.58 → $0.47−$0.11
  • maxCost per task$0.17 → $0.15−$0.026
  • single settingCost per task$0.57 → $0.51−$0.052

1 price change

Price changes 1

  • maxCost per task$0.49 → $0.57+$0.085

2 removed releases · 2 price changes

Price changes 2

  • highCost per task$0.011 → $0.014+$0.0022
  • highCost per task$0.092 → $0.11+$0.015

Removed releases 2

  • was 39.1 peakhad one setting
  • was 9.8 peakhad one setting

1 score change · 1 price change

Score changes 1

  • single settingIntelligence42.4 → 42.7+0.3

Price changes 1

  • single settingCost per task$0.093 → $0.080−$0.014

1 new release

New releases 1

  • peak 42.4one setting, $0.22 per task

1 new release

New releases 1

  • peak 45.1one setting, $0.26 per task

3 price changes

Price changes 3

  • maxCost per task$0.31 → $1.04+$0.72
  • maxCost per task$0.27 → $0.49+$0.22
  • maxCost per task$0.22 → $0.17−$0.045

1 price change

Price changes 1

  • single settingCost per task$0.95 → $0.57−$0.39

2 price changes

Price changes 2

  • single settingCost per task$0.40 → $0.95+$0.55
  • single settingCost per task$0.21 → $0.19−$0.020

1 new release

New releases 1

  • peak 30.3one setting, $0.40 per task

1 price change

Price changes 1

  • single settingCost per task$0.25 → $0.21−$0.042

6 price changes

Price changes 6

  • single settingCost per task$0.78 → $0.47−$0.31
  • highCost per task$0.11 → $0.092−$0.015
  • highCost per task$0.013 → $0.011−$0.0010
  • highCost per task$0.053 → $0.058+$0.0041
  • single settingCost per task$0.60 → $0.58−$0.021
  • single settingCost per task$0.038 → $0.038+$0.0001

1 new release

New releases 1

  • peak 26.6one setting, $0.71 per task

1 price change

Price changes 1

  • single settingCost per task$0.30 → $0.25−$0.052
Load older