ExoBrain
Epoch AIAI costsBenchmarks

The falling price of benchmarks

Getting the same result from AI is becoming more than ten times cheaper every year, faster than electricity, batteries or computing.

Joel Miller

Joel Miller

2 min read
The falling price of benchmarks

Our chart this week comes from Epoch AI. It estimates that the cost of reaching a fixed level of AI performance has fallen by about 47% per quarter since 2023, or roughly 13x per year. That is faster than the declines measured for electricity, lithium batteries, DNA sequencing and computing. Scoring 75% on the GPQA Diamond science exam cost around $0.30 per question with o3 in January 2025. Under 18 months later, GPT-5.6 Luna matched it for $0.0004.

The rate varies by task. Maths benchmarks show the steepest falls and games of skill the slowest. On SWE-bench Verified, which tests the software engineering work businesses increasingly give to agents, the cost has fallen by about 27.5% per quarter since mid-2024, or around 3.6x per year.

Benchmarked intelligence is clearly getting much cheaper. Complex outcomes often need far more tokens, because agents working on long tasks reason, retry and call tools many times over. The cost of a finished piece of work can therefore fall more slowly than the cost of answering a single benchmark question.

Subscribe to the ExoBrain Weekly Newsletter

Stay up to date with AI. Get analysis of the week's most important stories, plus a focused roundup across business, governance, research and infrastructure.

Follow us on LinkedIn