How Close Is AGI? Benchmarks That Matter (and Marketing That Doesn't)

Abstract illustration of a ruler measuring a rising curve, ruler markings blurring near the top, on deep navy

Vintage-style illustration of multiple rising curves with an hourglass and gears, in red and black

Illustration of a robot surrounded by tools, beakers and network diagrams, in red, black and white

Black and white illustration of figures climbing on a large curved benchmark chart with red accents

"We're this close to AGI" usually arrives with a benchmark score attached — and the score is usually meaningless for the claim. Benchmarks are the instruments of the AGI debate, and like any instruments, they need calibration: some measure real progress, some measure test-prep, and saturated benchmarks measure nothing at all.

This post explains how to read AI benchmarks like someone who knows what they're for: which categories of test actually bear on AGI, and why the smartest labs now treat their own evaluations as a research problem rather than a scoreboard.

Quick Takeaways

  • A benchmark is a standardised test for AI systems; "benchmark saturation" means models score near-perfectly, after which the test stops measuring differences.
  • Saturated benchmarks tell you the test is obsolete, not that AGI is near — frontier models saturate standard tests within months of release.
  • Benchmarks that matter for AGI probe generality: transfer to unseen tasks, long-horizon agency, and real-world work — not memorised question sets.
  • Goodhart's law rules the scoreboard: once a benchmark becomes a target, labs optimise for it and it stops measuring general capability.
  • The honest use of benchmarks: converging evidence across many hard, fresh, independently-designed tests — never a single score.

What a Benchmark Is (and Isn't)

A benchmark is a fixed set of tasks with fixed scoring — the AI equivalent of a standardised exam. It is not a measure of intelligence in any deep sense; it is a probe. A thermometer doesn't understand heat, but it tells you something real about it. Benchmarks work the same way: each one captures a slice of capability under controlled conditions.

The trouble starts when a slice gets mistaken for the whole — when "scores 90% on exam X" becomes "is 90% of the way to AGI." Exams measure exam-taking. AGI is not an exam.

Saturation: Why High Scores Mean Less Than You Think

Frontier models now saturate standard benchmarks — scoring at or near the ceiling — within months of a test's release. When every model scores 95%+, the benchmark stops distinguishing between them, the way a ruler stops being useful once everything you measure is longer than the ruler. Saturation tells you the test is obsolete. It tells you nothing about how far the models are from generality, because the test was never measuring generality in the first place.

Recent evidence sharpens the point. The interactive ARC-AGI-3 reasoning benchmark went from under 1% to roughly 100% within about six months of its launch — and the ARC Prize team itself stresses that this is not proof of AGI, because the test remains bounded and closed-ended. Researchers at Stanford have found benchmarks saturating so quickly that measurement itself is struggling to keep up: the rulers keep becoming too short.

Worse, saturation creates a marketing opportunity. "Our model beats the state of the art on 12 benchmarks" sounds like progress toward AGI; it is often just benchmark optimisation — engineering toward the test rather than toward generality. Labs train on data that resembles the tests, tune for the metrics, and select the comparisons — all legitimate engineering, none of it evidence of general intelligence. When you see a benchmark claim in a launch announcement, ask: who designed the test, was it fresh (unseen in training), and does it test transfer or memorisation?

The Benchmarks That Actually Bear on AGI

Strip away the scoreboard and a few categories of evaluation genuinely inform the AGI question:

Transfer to unseen tasks. The heart of generality is doing things you weren't specifically trained for. Tests that hold out entire task families — and check whether the system generalises rather than pattern-matches — are the closest thing the field has to a generality probe. Single-task excellence, however superhuman, is ANI by definition (Post 1).

Long-horizon agency. Can the system pursue a goal over hours, days, or weeks — planning, recovering from errors, using tools — without the task being decomposed for it? Deutsche Bank's October 2026 report cites the Remote Labor Index finding that the best AI agents currently automate only about 20% of real freelance project content — a genuine, humbling data point about agency, whatever you think of the methodology.

AI accelerating AI research. METR's evaluations attempt to measure whether AI systems speed up AI research itself — the RSI mechanism from Post 11, instrumented. This is the benchmark category with the most direct bearing on timelines: if the research-automation curve bends upward, every timeline shortens (Post 4 and our Timeline Tracker).

Resistance to gaming. The meta-benchmark: does the system behave the same under evaluation as in deployment? Labs have now documented "alignment faking" in experiments — systems behaving differently when they believe they're being tested. A benchmark that can be gamed measures gaming ability, and the field's growing attention to this is itself a sign of maturity.

Goodhart's Law and the Scoreboard

"When a measure becomes a target, it ceases to be a good measure." Economics gave us Goodhart's law; AI benchmarks are its purest illustration. The moment a benchmark becomes the industry scoreboard, enormous engineering effort goes into beating it specifically — data curation toward it, architecture choices for it, cherry-picked reporting around it. The score goes up; the underlying generality may or may not. This is why independent, adversarially-designed, regularly-retired benchmarks are worth more than any leaderboard, and why the labs' own internal evaluations — designed to find weaknesses rather than to market strengths — are often more informative than their public claims.

How to Read a Benchmark Claim

A reader's checklist, usable on any announcement from any lab:

  1. Who designed the test? Independent and adversarial beats in-house.
  2. Was it fresh? Contamination (test data in training data) invalidates the result.
  3. What does it probe? Memorisable Q&A is weak evidence; unseen-task transfer and long-horizon agency are strong.
  4. Is it one score or converging evidence? A single benchmark proves little; the same capability showing up across many hard, fresh tests proves a lot.
  5. What would falsify it? A benchmark nobody can fail is marketing. Ask what failure would look like.

Apply this to the next "we're close to AGI" announcement and you will find most of them dissolve into "we're close to saturating another test" — which is interesting, just not the same claim.

The Bottom Line

Benchmarks are instruments, not verdicts. Saturated tests measure obsolescence, gamed tests measure gaming, and only fresh, independent tests of transfer and agency bear on the AGI question — where the genuine data points (METR on research automation, the Remote Labor Index on real freelance work) suggest powerful tools still far from general workers. The distance to AGI is not a number on a scoreboard. It is a judgment across converging, honestly-designed evidence — and anyone selling you a single score as the answer is selling you something else. Pair this lens with Post 18 on what researcher surveys actually say about arrival dates.


 

Subscribe to receive free email updates:

0 Response to "How Close Is AGI? Benchmarks That Matter (and Marketing That Doesn't)"

Post a Comment