OpenAI Boosts AI for Legacy Code

Bridging the Legacy Code Gap

OpenAI's announcement of significantly improved AI capabilities for low-resource programming languages like COBOL, Fortran, and Visual Basic is a crucial step towards modernizing critical infrastructure. The performance gains, especially in migration accuracy (97.9% at higher reasoning effort) and the ability to build complex systems like SQLite from scratch (87.4% average pass rate), are remarkable. The cost reduction is also a key enabler for wider adoption. The transparency in releasing evaluation methodologies (MultiLangBench and the COBOL migration evaluation) and making the benchmark publicly available is commendable, fostering community engagement and allowing for independent verification. This advancement directly addresses a well-known pain point for many enterprises struggling with aging codebases, offering a potential lifeline for maintenance, modernization, and migration efforts.

However, several considerations warrant attention. While the reported accuracy figures are impressive, the practical application in production environments still requires significant human oversight and rigorous testing, as OpenAI itself notes. The 'byte-for-byte parity' for migration is a high bar, and ensuring that subtle business logic nuances, especially those deeply embedded in COBOL's fixed-point arithmetic or specific error handling, are perfectly preserved across different language paradigms remains a complex challenge. Furthermore, the 'low-resource' nature of these languages implies a scarcity of training data. While the models are showing progress, the long-term maintainability and evolution of these AI capabilities will depend on continued access to high-quality, diverse datasets. The focus on specific languages is beneficial, but the broader implications for other under-resourced or domain-specific languages are yet to be explored. The potential for 'hallucinations' or subtle bugs introduced by AI in critical financial or safety systems, even with high reported accuracy, is a persistent concern that demands robust validation processes. Ultimately, while this is a significant leap, it's a tool to augment human developers, not replace them, especially in high-stakes scenarios.

Key Points

• OpenAI has significantly advanced its AI models to better handle low-resource programming languages like COBOL, Fortran, and Visual Basic.
• New models show substantial improvements in building software from scratch (e.g., SQLite engine) and migrating existing codebases while preserving behavior.
• GPT-6 Astra achieved an 87.4% average pass rate on MultiLangBench across 20 languages, a significant jump from GPT-5.6 Sol's 26.8%.
• COBOL migration accuracy reached 97.9% at higher reasoning effort with GPT-6.1 Sol, compared to 5.4% for GPT-5.6 Sol.
• Cost-efficiency has also improved, with GPT-6 Luna offering a 78% lower API cost for comparable gains on MultiLangBench.
• The company is releasing the MultiLangBench evaluation setup and test suite to the public.

---

📖 Source: Advancing coding in low‑resource programming languages