Artificial Analysis 发布 AA-3D 基准，评测 AI 模型构建交互式 3D 场景的能力

All articles

AA-3D measures how well AI models build interactive 3D scenes that people want to explore. Each model works as an agent in an offline sandbox and turns a written brief into a three.js web page. Our recruited panel compares pairs of scenes built from the same brief, without knowing which model made which, and we rank the models by Elo. Claude Sonnet 5.5 is first at 1194, statistically tied with three other Anthropic and OpenAI models.

AA-3D v1.0 leaderboard

AA-3D Elo

Claude Sonnet 5.5 has the highest Elo, 1194, but the top four are statistically tied: their 95% confidence intervals all overlap with that of Claude Sonnet 5.5.

All four come from Anthropic and OpenAI. Meta’s Muse Spark 1.3 is seventh at 1035. The best-placed open weights models, Kimi K3 (982) and MiMo-V2.6-Pro (977), are 11th and 12th, more than 200 points behind.

For up-to-date results, see the AA-3D leaderboard.

Try it yourself

Explore scenes built by two different models for the same brief, then pick the one you prefer to reveal which model built it. Votes cast here do not count toward the leaderboard.

How AA-3D works

Each model works as an agent in an offline sandbox. It gets a written brief for one interactive scene, plus a delivery contract that every task shares. It writes code, renders the scene, looks at its own screenshots and revises. The sandbox has Blender, Rapier physics, Python, a headless browser and a library of CC0 assets. A run stops after three hours or 250 turns, and the result is an index.html page that runs the scene in a browser.

Watch a scene being built

GPT-6 Astra has its first complete page after 6 minutes. It finishes at 68 minutes. Claude Sonnet 5.5 shows nothing until minute 42 and keeps working until 2 hours 58 minutes; Claude Sonnet 5.5 writes 6.2 times as many output tokens as GPT-6 Astra, but costs only 1.3 times as much. Its run costs $14.93 against $11.62, and its list prices are a fifth of GPT-6 Astra’s

Explore the scenes

Pick a domain, a task and a model to load the scene that model built.

Task

Model

Create an interactive adult man performing weightlifting exercises, with views that expose his muscles, skeleton and organs in anatomically accurate form.

21 of 21 models have a scene for this task in blind voting.

The cost of an AA-3D scene

AA-3D Cost per Scene

Inkling is not shown because we have no representative cache hit rate for it.

AA-3D Elo vs. Cost per Scene

Inkling is not shown because we have no representative cache hit rate for it.

What AA-3D tells us about building in 3D

The top models look at their own work early

The top five first look at an image of their own scene after a median of 10 to 13 turns, and by turn 20 in at least 89% of their scenes. Models ranked 14th to 20th wait until turn 15 to 40. Across the 19 models that can view images, looking early tracks Elo closely (Spearman 0.82). The top five also start testing in the browser sooner, at a median turn of 16 against 25. How often a model looks matters less than when. MiniMax-M3 looks at its own scene 54 times per scene, more than any other model, but by turn 20 in only 7% of its scenes. It ranks 19th.

AA-3D Elo vs. Early Visual Inspection

The chart shows the 19 models with an image view tool. GLM-5.3 and Nemotron 3 Ultra are text-only and built their scenes without seeing them.

Detailed scenes win more votes, and stronger models build them

The most detailed fifth of scenes in each task has screenshots twice as large (168 KB against 84 KB) and a third as empty (22% against 60%). It wins 63% of its votes, against 31% for the least detailed fifth. The models’ Elo alone predicts 61% and 33%, so the detailed scenes come mostly from the stronger models. These data cannot show whether detail is part of what makes those models strong. Within one model, the effect is small: a scene one standard deviation more detailed than the model’s other scenes wins about 1.5 points more of its votes (1.3 to 1.8, depending on the detail measure). Of 86 scene measures, 11 pass correction: the detail measures, render resolution and antialiasing, GPU time per frame and the number of page elements. None moves votes by more than 4 points.

Detail combines four screenshot measures, defined in What Detail Measures. Within the same model and task, one standard deviation more detail adds 1.5 points (p = 0.001, q = 0.011). 1,008 scenes, excluding 23 that break on the voting page.

OpenAI's GPT-6 models build better 3D scenes than their general intelligence predicts

General intelligence goes a long way in AA-3D. Each point on the Artificial Analysis Intelligence Index is worth about 18 Elo (Pearson 0.92), a closer fit than Terminal-Bench 4.0 (0.81) or SciCode (0.76). GPT-6 Luna and GPT-6 Sol beat that trend by more than any other model, by 141 and 93 Elo, and GPT-6 Astra beats it by 46. GLM-5.3 falls furthest below, by 197. It cannot view images, which may explain part of the gap: Z AI’s GLM-5.3-Flash, which can, scores 149 higher with a lower Intelligence Index. Nemotron 3 Ultra, also text-only, sits near the trend.

AA-3D Elo vs. Intelligence Index

We use Pearson’s r where a relation is linear on the chart’s axes, and Spearman’s rank correlation where one model would dominate r or a measure takes only a few values. We call a correlation not significant when its 95% confidence interval includes zero.

Most scenes use the asset library, but the assets do not win the votes

Three in four scenes use something from the CC0 library, as the shared instructions encourage, and models that use it more often tend to rank higher (Spearman 0.53). But a model’s library scenes win no more often than its other scenes, once we account for the opponent. GPT-6 Luna uses the library in only a third of its scenes and still ranks eighth.

We sort every tool call into five groups by purpose, then into subgroups by command. Browser tests either save a screenshot or only read errors. Code calls write or patch files, or read files and the bundled three.js docs. Asset calls browse the CC0 library, process images with Python or run Blender. Image views show the agent its own work (mostly page screenshots, also contact sheets and textures), an asset preview or a reference image from the brief.

AA-3D Elo vs. UI Controls

Explore the full results on the AA-3D leaderboard.

Inside an agent run

Pick a task to replay every model’s run on a shared clock. Hover over a lane to see each action, and click it to see where the time went.

GPT-6 Sol is the quickest of the top five on all six tasks. Its median run takes 60 minutes, against 115 to 151 for the other four; The top five do not test more than the others. As a group they spend 25% of their time on browser tests, about the same as everyone else; Every top-five run looks at its own scene. GLM-5.3 and Nemotron 3 Ultra are text-only, so none of their 12 runs does

What's next

We will add new models to the AA-3D leaderboard as they are released.

AA-3D resources

The AA-3D leaderboard has the full results, updated as new models are released; Try AA-3D Voting to compare scenes yourself; The methodology page documents the generation settings, the scene checks and how we compute Elo; AA-3D runs on Stirrup, our open-source agent harness