Blog

If AI is going to improve itself, who writes the test?

This research explores whether AI can create its own tests, and if humans still have a role in that process.

As AI systems get better at improving themselves, a key question arises: can they also design the benchmarks—the tests and challenges—used to measure their own progress? Creating good benchmarks is hard, creative work currently done by human scientists. It involves deciding what to measure, gathering source material, and designing tasks that are difficult but fair.

The researchers built a system calledmark to see if an AI agent could handle this entire process on its own, and to find out where humans might still be needed.

The AI agent runs in a loop. It proposes a benchmark, which includes creating tasks and a reference solution. Other AI “solver” agents then attempt these tasks, and their scores tell the creator if the benchmark is hard enough. A separate AI “judge” reviews the benchmark for quality, checking things like whether it’s valid, solvable, and actually measures what it claims to. The agent uses all this feedback to revise its benchmark over several iterations.

The main finding is that humans still matter, but only when they give concrete, detailed guidance.

- No Human Help: When the AI worked completely alone, it created benchmarks that were too easy—solver AIs scored above 80 out of 100, meaning the tests were “saturated” and didn’t reveal much about model capabilities.