A culture of silence around AI mistakes
Nobody enjoys confessing to an error, and organisations enjoy it even less. Research published in the Journal of Business Ethics on risk reporting in financial services found that fear, embarrassment and guilt should not be understated in decisions about disclosure. A mistake caught before it reaches a client causes no visible harm, while admitting it can invite scrutiny and reputational damage that reaches far beyond the original error. For an individual firm, staying quiet can look like the rational response.
The result is a sector sitting on a large stock of collective experience it never shares. Every AI implementation teaches its owner where models perform poorly, which controls prevented a mishap and when a human needed to override the machine. Held in isolation, such knowledge helps one firm. Pooled, it could raise the safety standards of the entire industry.
The aviation precedent
Financial services does not need to invent a solution from scratch. This year marks five decades of NASA’s Aviation Safety Reporting System, a voluntary and confidential scheme that has collected more than 2.3 million reports from pilots, controllers and engineers. Each report is analysed and the lessons are circulated across the aviation community, so one airline’s near miss becomes every airline’s warning. The system has endured because confidentiality removes the penalty for honesty.
Lord Clement-Jones, a leading parliamentary authority on AI, has warned that the pace of technological advancement could outrun society’s capacity to govern it. Aviation solved a comparable problem by making disclosure cost-free. HM Treasury is now proposing that financial services try the same principle, through a confidential and voluntary repository where firms can log AI incidents and near misses without exposure.
Removing the fear of attribution
Confidentiality is a step in the right direction, since restricting reports to a select group reduces the risk of fallout. The embarrassment problem needs a stronger answer. Applying Chatham House Rules would mean no entry in the repository could be traced back to a particular firm, removing the reputational incentive to keep quiet altogether. If the fear of attribution is designed out, a near miss becomes an opportunity to improve rather than a confession to conceal.
Where advantage ends and collective resilience begins
Innovation is private, governance is shared
The more fundamental objection is commercial. Firms spend considerable time and resources refining governance structures and learning how to deploy AI responsibly, and handing over such lessons to competitors can feel like self-sabotage. Conventional wisdom holds that hard-won insight should be protected as a source of edge rather than circulated for the common good.
The counterargument draws a clear line between technological innovation and governance. Better AI models can create competitive advantage; better AI governance should not. A safeguard discovered by one bank protects its clients, but shared widely it protects every bank’s clients at no cost to the original firm. Responsible AI use, in short, is not a zero-sum game.
A proactive shift for financial services
The proposal also marks a departure in how the industry learns from risk. Governance frameworks in financial services have historically arrived in the aftermath of collapses and bubbles, imposed or accelerated once the damage was done. The Financial Services AI Adoption Plan takes a proactive line instead, aiming to capture early warnings before they culminate in a crisis. A repository of near misses would give that ambition practical substance, letting firms adapt to evidence from their peers rather than waiting for a catastrophe to force their hand.
Learning before failure, not because of it
The obstacle to safer AI in financial services is ultimately cultural rather than technological. Three quarters of the sector already runs on the technology somewhere, generating a steady stream of operational lessons that almost nobody shares. HM Treasury’s 75% adoption figure shows how quickly the industry has committed to AI, and how much unrecorded experience now exists alongside it.
If the repository succeeds, financial services will begin absorbing the lessons of its mistakes before any of them become public disasters. Chatham House Rules, guaranteed confidentiality and a clear separation between proprietary models and shared governance would all strengthen the design. If embarrassment wins instead, the industry risks learning about its AI failures the traditional way, from the front pages.