In a November 19 blog post OpenAI described Performance Factory, an internal system where a performance alert opens a task, Codex agents find the root cause, a separate Codex session runs an adversarial review, Codex opens a fix PR for an engineer to review, and an agent later checks that metrics recovered. Its first workflow, the Regression Workflow, tracked more than 1,700 investigations and root caused over 600 cases in just over three months, about 17 per day; in a manual review of 100 root cause suggestions, 70% were graded accurate.
Key Takeaways
- ✓Official: OpenAI blog post on November 19 by Members of Technical Staff Okay Zed and Cyrus Forbes, also announced by @OpenAIDevs
- ✓Architecture: four components, a task tracker with configurable workflows, a dispatch loop, an API that runs Codex in the cloud, and subscriptions that wake agents on events such as PR comments or completed deployments
- ✓Numbers: 1,700+ investigations and 600+ root causes in just over three months, about 17 per day; of 100 manually reviewed root cause suggestions, 70% were graded accurate and 30% needed refinement or did not fully establish causality
- ✓Guardrails: both diagnoses and fixes pass an adversarial review by a separate Codex session, and engineers still decide whether to merge each PR
- ✓Next: synthetic performance tests so agents can measure fixes before review, and exploring shipping low-risk fixes without a human in the loop

Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
Codex has sped up ChatGPT development, and faster shipping means more performance regressions. Engineers used to dig through metrics, logs, traces and code by hand, and the team built more than 30 performance tools, yet still could not keep pace. OpenAI turned that process into an automated system called Performance Factory.
How it works
The system has four parts: a task tracker with configurable workflows, a dispatch loop that assigns ready tasks to agents, an API that runs Codex in the cloud, and subscriptions that wake agents when a PR gets a review comment or a deployment completes. State and findings live in the task, not in any one agent's context. The investigation playbook is captured in a perf-forensics skill, and both diagnoses and fixes go through an adversarial review by a separate Codex session.
Benchmarks
OpenAI published no benchmark scores, only operating data: in just over three months the Regression Workflow tracked more than 1,700 investigations and root caused over 600 cases, about 17 a day. In a manual review of 100 root cause suggestions, 70% were graded accurate and 30% needed refinement or did not fully establish causality. OpenAI also says getting fixes right is harder than finding causes, since agent changes are sometimes overcomplicated or introduce new issues.
What developers should do
The pattern is easy to borrow: encode your team's debugging playbook as a skill, keep state in a task record that agents and engineers can hand off, let agents sleep while waiting for reviews or deploys and wake them on events, have a separate session challenge key conclusions, and keep merge decisions with humans.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.