Arena’s interaction judge tests what agents actually finish

Static screenshot graders are no longer enough to tell which AI model builds working software. Arena published two open datasets alongside new browser-level evaluation results showing that testing live user interactions pushes agent pass rates up by 20 percentage points — just as Claude Fable 5.2 cements its #1 lead on the AI leaderboard.

For months, web-app benchmarks graded generated interfaces by inspecting DOM snapshots or rendered PNGs. While visually neat, those checks missed broken checkout flows, dead state handlers, and modals that refused to close.

Inside ArenaWebdevPreferences and Correctness

To close that gap, Arena released ArenaWebdevPreferences (recorded side-by-side human interactions inside deployed WebDev battles) and ArenaWebdevCorrectness, a benchmark spanning 189 real web apps and 362 explicit behavioral requirements verified by autonomous browser judges.

Key findings from the technical report and live leaderboard telemetry include:

• Self-repair jumps 20.1 points: Allowing coding agents to inspect runtime browser failures through interaction feedback boosted requirement pass rates from 55.4% to 75.5%.
• RL on live interaction beats offline reward: Training Qwen-3.5-9B with an interaction-trained reward model added +50 Elo compared with standard static preference baselines.
• Claude Fable 5.2 dominates WebDev and Overall: On Vigas Arena rankings, anthropic/claude-fable-5.2 leads Overall at 30.48% win share and WebDev at 1838 rating, ahead of Claude Opus 5.5 (19.51% overall) and Gemini 4 Carbon (16.90% overall).

Why token price and eval fidelity now converge

High leaderboard ratings increasingly depend on multi-turn execution loops — build, click, inspect console errors, patch — rather than one-shot prompt luck. That shifts unit economics directly onto mid-cost models with strong agentic tool use.

At $1.80 per 1M input tokens and $9.00 per 1M output tokens on the Vigas model router, Claude Fable 5.2 undercuts flagship Opus 5.5 ($15 / $75) by more than 8× while still beating it on both Overall and Coding win shares. Builders running continuous browser-verification loops can also pair Fable 5.2 with high-speed fallbacks like google/gemini-4-carbon without rewriting client code.

What engineering teams should change now

If your team ships coding agents or generative UI apps from our AI tools directory, move evaluation earlier into headless Playwright or Chromium smoke runs instead of gating releases purely on LLM text diffs. Capturing click paths, runtime exceptions, and state persistence closes the exact failure gap exposed by Arena’s new open benchmarks.