OpenAI Solves Legacy Software Migration: MultiLangBench Lifts COBOL Pass Rate to 97.9%, Modernizing Enterprise Stacks
OpenAI officially announced breakthroughs in building and migrating software written in low-resource and legacy languages (COBOL, Fortran, Visual Basic, OCaml), introducing the MultiLangBench evaluation suite. Tackling the trillions of dollars in critical banking and enterprise mainframe infrastructure, the benchmark challenges models to synthesize a SQLite-compatible database engine across 20 distinct languages. GPT-6 Astra boosted average test pass rates from 26.8% to 87.4%, with COBOL soaring from 10.9% to 72.1%. In enterprise COBOL migration tasks, GPT-6.1 Sol achieved a 97.9% success rate while cutting inference costs by 39%.
Key Takeaways
- OpenAI unveiled breakthroughs in low-resource and legacy languages with MultiLangBench, challenging models to implement database engines across 20 languages
- GPT-6 Astra raised the average pass rate from 26.8% to 87.4% (clearing >83% in 19 languages), with COBOL hitting 72.1% and Fortran 83.5%
- In enterprise COBOL migration benchmarks, GPT-6.1 Sol scored 97.9% (vs 5.4% for GPT-5.6 Sol) while reducing execution costs by 39%
- OpenAI confirmed open-sourcing the MultiLangBench test suite on GitHub to accelerate automated modernization of mainframe software
