ArtifactArena: Evaluating Models by What They Build in the Physical World

We benchmark these capabilities with an Elo ranking of frontier models derived from head-to-head tournaments between their artifacts. It comes from a paper posted to arXiv on 26 October 2026. To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce ArtifactArena, an open-ended platform where models face a physically grounded hardware-software co-design challenge: engineering fully functional robots to compete in a simulated arena. We evaluate a frontier model's zero-shot, verifier guided refinement, and open-ended physical design capabilities through three harnesses that refine their bot artifacts based on text descriptions, physics simulator feedback, and gameplay data. By releasing this framework and tournament infrastructure for ongoing community submissions, we establish a living, non-saturating testbed to continuously measure the expanding limits of open-ended intelligence in the physical world. Please visit https://artifactarena.ai for more information.

Why it counts

We benchmark these capabilities with an Elo ranking of frontier models derived from head-to-head tournaments between their artifacts.

Sources

Primary source: primary source

What is not known

This brief does not claim independent replication. Claims that appear only on X and not in the primary source stay unknown.