How OpenAI Tests New Models: The Challenge of Rewriting Codex in Rust
OpenAI's Codex lead Andrew Ambrosino revealed the internal benchmark for new models: fully rewriting the desktop Codex app in Rust and GPUI.
Andrew Ambrosino, lead of Codex at OpenAI, shared how the company internally evaluates new models. Every time a new model is released, he assigns it the same task: fully rewriting the desktop Codex application in Rust using the GPUI framework.
The goal is not just to create a demo, but to build a complete native application that works consistently across different platforms. According to Ambrosino, the complexity lies not only in the code but also in coordination. The Codex team changes the product so rapidly that the model literally cannot keep up with a fixed version.
The idea is for one sufficiently powerful model to simultaneously support the Mac version on AppKit, the Windows version on WinUI 3, and a separate full version in Rust.
"We're not there yet. But we're much closer," he noted.
Why it matters
AnalysisThis approach highlights that evaluating AI capabilities requires complex engineering tasks involving cross-platform native application maintenance, beyond standard benchmarks.
Discuss in community
Share your questions and insights with developers
Related in Ecosystem
Practical guides, core terminology, and agent skills
Codex Skills for Beginners: Automating AI Agent Workflows
A comprehensive hands-on guide to building and executing Codex Skills: SKILL.md anatomy, explicit vs. auto-triggering, skills vs. scripts vs. tools, Skills API, and safety boundaries.
OpenAI launches retro console for vibe coders
OpenAI has released ModRetro Chromatic Codex Edition, a $200 Game Boy-compatible console with a rewritable cartridge and a Codex plugin for building games.
OpenAI GPT (Flagship Models of the GPT Series)
The primary universal line of large language models from OpenAI (GPT-4, GPT-4o). Optimized for complex text analysis, programming, creativity, and daily intellectual tasks.
ChatGPT Updates: Plugin Development and Codex Workflows
Comprehensive engineering guide to creating, packaging, testing, and submitting plugins for ChatGPT and Codex: manifest specification, MCP servers, lifecycle hooks, custom marketplaces, and OpenAI review.