Method
The arena
Each prompt is given verbatim to every system — Claude Code on the Anthropic side, Codex CLI on the OpenAI side, plus archived ChatGPT web-app runs from April 2026. One attempt, no retries, no hand edits. The output must be a single self-contained HTML file that works offline. What the model produced is what you see, including its mistakes.
Clean-room runs
Every run made from 2026-08-02 onward happens in an empty folder with project instructions, memory, hooks, skills, and plugin context switched off, so the model sees the prompt and nothing else — no house style, no sibling builds from rival models, no knowledge of this site. Isolation is verified with a probe before a batch runs. Earlier runs (the Opus 4.8, Codex 5.5, and ChatGPT Pro artifacts) were not made under these controls, so treat cross-generation comparisons with that caveat.
The runner instruction
One identical instruction is appended to every prompt for every model, because the runners are non-interactive: work autonomously and do not ask any clarifying questions; where the brief is silent, make a reasonable engineering decision and continue; write the complete result to index.html; deliver the finished file in this single turn. Without it, some models stop to ask a question and produce nothing. It is the same text for every contestant.
The bench
33 trials across Python, JavaScript, Delphi, and C#. Each trial is a task folder with a written specification and a locked test suite (624 tests in total). Six models run the same trial with one attempt each. VERIFIED means every locked test passed and the run record matches the preserved sandbox. Diffs and agent reports are published as generated.
Sides and colors
Claude / Anthropic is always Side A in copper. Codex / OpenAI is always Side B in steel blue. Archived third-product runs are pewter. Verdigris green marks a verified pass; oxblood marks a failure. Color never decorates — it identifies.
Where numbers come from
Everything under Arena and Bench is measured on this site. The Board additionally reproduces coding results from public leaderboards; those always carry the source name, a link, and an as-of date, and are never merged with our measurements. News items link their sources; green items are our own run log.
Limits
One attempt per model per task means n=1 in every cell: results are evidence, not statistics. Thinking effort is not matched across vendors — Claude models ran at the CLI default, Codex models ran at reasoning xhigh. Each build states its own setting, and that gap is a real advantage to the Codex side on hard prompts. Timing depends on infrastructure load at run time. We state these limits wherever the numbers appear.
Raw bench data: bench/gauntlet-001.json