Big Tech’s AI auditors will only see what they’re shown

Last week, six of the most powerful AI companies in the world promised the White House they would let outsiders check their work.

On Sept. 29, Nvidia, OpenAI, Anthropic, Google, Meta and xAI signed an accord on the most advanced AI models that U.S. President Donald Trump called “morally binding.” Each company has promised it will “partner with an independent external auditor or evaluator” and have independent board members review the auditors’ reports and make sure problems get fixed.

Outside audits are a good idea, and I’m glad to see them in there. But the accord says nothing about who chooses the auditors, what they can examine or whether any findings are made public; there’s no penalty for ignoring it. As it stands, each company picks its own auditor and decides what that auditor gets to look at.

In 2000, Enron paid accounting firm Arthur Andersen — once part of the so-called “Big Five” of international accounting firms, now the “Big Four” after Arthur Andersen’s implosion — $25 million to audit its books and another $27 million for tax and consulting work. Within two years, both firms had collapsed. Congress responded with the Sarbanes-Oxley Act, which took the choice of auditor away from management, stopped auditors from selling most other services to the companies they audit and set up a public board to inspect the auditors themselves.

Andersen failed not due to a deficit in expertise, but in an incentive to look hard at a client paying it $52 million in a single year. Voluntary pledges in the AI industry don’t have a great record, either. In 2023, the leading AI companies made a set of voluntary commitments to the White House, including letting outside experts test their systems before release. When researchers at Stanford, Harvard and Brown went back and scored 16 companies against those commitments, the average came to 53%. On the promise to help outsiders find and report problems, eight of the 16 scored zero.

Even when outside testing does happen, the developer sets the terms. METR, a nonprofit that tests AI models, wrote that its testing of OpenAI’s o3 model before release was “conducted over only three weeks,” with “limited access to information about the model” and no view of the model’s step-by-step reasoning.

To be fair to OpenAI, it allowed the testing in the first place. But it shows how much rides on what a company chooses to share.

A trained AI model is essentially a very large file of numbers, often costing more than $100 million to produce, and whoever has a copy has the model. Handing copies to outsiders is a security risk. Until recently, that left testers with an awkward choice, which Google DeepMind has described well: either the testers hand over their questions, so the company sees the exam in advance, or the company hands over its model.

In August, DeepMind showed a way around that. Working with Singapore’s AI Safety Institute and three other outside groups, it ran what it called the world’s first double-blind test of an AI model, on the same principle as a double-blind drug trial. The test ran on sealed computer hardware, so the outside testers never saw the model and Google never saw their questions. Google says each side can check mathematically that its material stayed private.

It also moves the trust onto the hardware. Last year, researchers broke into this kind of sealed hardware with less than $1,000 of equipment, although they needed physical access to the server to do it. That’s worth taking seriously. But it’s a much smaller problem than “we can’t let anyone near the model,” and it can be managed by being careful about who controls the machines.

For our report, “Alternatives to Concentrated AI,” published this week by the Decentralization Research Center, my co-author Kyle Bligen and I interviewed 36 people and mapped nearly 30 working alternatives to concentrated control over AI. Our main finding was this one: outsiders can now scrutinize advanced AI without taking possession of the model or its data.

If the accord’s audits are going to be worth anything, auditors should choose what they test, now that they can do so without taking a copy of the model. Companies should also say who their auditors are and what the audits covered, even if the detailed findings stay private. Public companies have named their auditors for decades without giving away trade secrets. And the board committees reading these reports should answer to someone outside the company. Sarbanes-Oxley created an inspector for the inspectors. The accord has nothing like it, and Congress should decide what fills that gap.