OpenAI tests COBOL migration as Brazilian banks modernize

A close-of-day process that reconciles correctly, finishes within its allotted window and recovers from failure delivers tangible value. For COBOL practitioners, modernization starts with understanding that outcome and what it takes to preserve it. A language change may help, but it needs to address a business problem.

OpenAI's October 19 research adds relevant evidence to this discussion: improved COBOL migration results and an open coding benchmark spanning 20 languages. The opportunity deserves attention from the people maintaining these systems, including where keeping COBOL remains the best decision.

What OpenAI's evaluation establishes

OpenAI reports 86.6% accuracy for GPT-6.1 Sol at low reasoning effort and 97.9% at a higher setting on its migration evaluation. Success requires passing every hidden test with byte-for-byte matching observable outputs. These are vendor results; MnzAI Labs has not reproduced them.

The MultiLangBench repository covers a different experiment: implementing a SQLite-compatible engine, rather than migrating a banking core. Its README describes tests derived from SQLite 3.40.1 and recommends a 272-case subset to keep costs manageable. Comparisons need to disclose which set ran. Releasing this suite does not automatically make the separate migration evaluation reproducible.

For specialists, behavioral equivalence is the significant requirement. IBM's documentation of ROUNDED explains why scale and discarded decimal places matter. Converted code can compile successfully and still produce a different cent. Preserving behavior requires understanding declarations, operations and compiler options alongside the approved business rule.

Where Brazilian banks still work with COBOL

Recent public evidence documents continuing COBOL work. Its scope varies, and it cannot establish the share of each bank's processing that depends on the language.

Banco do Brasil provides useful context for retaining established workloads. A 2022 technical account written by one of its engineers describes an investment portfolio backend using COBOL, JCL and DB2. The author explains the environment's efficiency for large files and bulk loading. For a new Open Banking requirement, the team selected Quarkus and Kafka. This historical example illustrates a workload-specific choice that retained useful existing capabilities.

At Bradesco, an IBM case study published in May 2020 links the platform to resilience, security and integration with distributed services. It is historical supplier material with customer testimony, not a current infrastructure inventory.

Those qualities come from the whole system: language, platform, transaction manager, data, operations and people. COBOL and mainframe are not interchangeable terms. The recovery guarantees of a CICS unit of work do not transfer to a distributed application simply because its business logic has been translated.

Where AI can change the decision

Modernization is already under way. GFT describes an Itaú project using Batch2Cloud and Wynxx, reporting up to a 70% reduction in migration time during the initial phase. This is a partner-reported measurement without enough methodology to forecast the same savings elsewhere. AWS also describes a portfolio approach in which some applications remain on mainframes.

In MnzAI Labs' assessment, AI could lower the effort required to understand code, document dependencies, propose tests and explore alternatives. That can make application decisions better informed, including a decision to retain the current system. The business case needs to compare practical paths:

• Improve the existing environment: where service levels and total cost meet requirements, with supported compilers, testing, observability and skills development.
• Integrate or extract a capability: where the constraint lies in an interface or a bounded domain. Include latency, consistency and the burden of operating both environments.
• Migrate or redesign: where a demonstrated benefit in delivery time, cost, capacity or risk justifies conversion, testing, coexistence and future operations.

A useful proof of concept would begin with a bounded routine and authorized data, using reference outcomes reviewed by the business. Beyond amounts and files, I would require evidence on job restarts, duplicate processing after failure, commits, concurrency, reconciliation and performance within the actual processing window. Existing behavior is not automatically the desired specification: known defects and new requirements need approved, traceable changes.

The people who understand copybooks, job dependencies and production exceptions are essential to defining that proof. They can start with a specific question: which application is so difficult to understand and test that a needed business improvement remains blocked? That is a useful place to measure what AI contributes.