AI Finally Wants Independent Review. Now We Have to Decide What “Independent” Actually Means.

For years, the artificial intelligence industry has talked about safety, governance, transparency, alignment, and responsible deployment. Now something important is changing. The conversation is moving toward independent review.

That is progress, but it also creates a harder question that can no longer be avoided: what actually makes an AI evaluation independent?

It is not enough to hire an outside organization. It is not enough to put the word “independent” in a press release. It is not enough to allow an evaluator to inspect a system if the company being evaluated still controls what evidence can be seen, what scope can be examined, how findings can be characterized, what can be disclosed, or whether an unfavorable result can ever meaningfully reach the people who depend on it.

This question is no longer theoretical. Reuters reported on October 13 that OpenAI and Anthropic are negotiating with independent evaluators over how much control AI developers should retain over publication of safety findings. The dispute is not simply about whether outside review should occur. Both companies have publicly supported independent evaluation. The more difficult issue is what happens afterward, particularly when findings intersect with intellectual property, trade secrets, security-sensitive information, redactions, and editorial control.

Those concerns are legitimate. A serious evaluation process cannot require companies to expose proprietary model details, confidential customer information, exploitable vulnerabilities, or protected intellectual property to the public simply for the sake of appearing transparent. But there is also a line between protecting sensitive information and controlling independent scrutiny.

That line may become one of the most important governance questions in AI.

Independence is easy to claim when the result is favorable. The real test is whether the institution remains independent when the client hates the answer.

External Is Not the Same as Independent One of the easiest mistakes to make in this emerging market is treating the words “external” and “independent” as interchangeable. They are not.

An evaluator can be outside the organization and still be structurally dependent on it. If the evaluator relies heavily on repeat business from the company being assessed, economic pressure exists. If the client determines which systems or behaviors can be examined, scope pressure exists. If the evaluator only sees evidence the client chooses to provide, evidence pressure exists. If findings can be rewritten or suppressed before publication, disclosure pressure exists. If the evaluator can identify a problem and then sell the remediation needed to fix it, another conflict appears.

None of those factors automatically proves misconduct. That is not the point. The point is that independence cannot depend on everyone behaving perfectly.

Mature assurance systems do not assume that incentives disappear because everyone involved has good intentions. They design around those incentives. That is why financial relationships, ownership interests, consulting arrangements, evaluator competence, recusal procedures, and decision authority matter.

AI will not be different simply because the technology is newer. If anything, it will require more discipline because the technical complexity, commercial stakes, security risks, and information asymmetry are unusually high.

We should expect companies to protect their intellectual property. We should expect evaluators to protect their reputations. We should expect investors to care about valuation. We should expect sales organizations to care about customers. We should expect regulators to care about public safety.

The governance system has to work even when those incentives collide.

Financial Independence The first test is financial. Who pays the evaluator, and what exactly are they paying for?

Most independent evaluation models will still involve the company being evaluated paying for the work. That alone does not destroy independence. Entire assurance industries operate under similar structures.

The problem begins when payment becomes linked to the result.

If the evaluator earns more when the client passes, independence is weakened. If certification fees are contingent on a successful outcome, independence is weakened. If negative findings create an opportunity for the same evaluator to sell remediation, implementation, consulting, software, or other downstream services, the incentives become more complicated.

Again, that does not mean the evaluator will act improperly. It means the institution has created an incentive that should have been designed out.

A credible assurance model should be economically sustainable without making success more profitable than failure. The client should be paying for the examination, not for the answer.

That distinction sounds simple, but it changes the entire structure of the relationship. It also means accepting something that can be commercially uncomfortable. Sometimes the company paying you has to receive a result it does not want.

If the business model cannot survive that moment, the independence model was never strong enough.

Evidence Independence The second test is evidence. An evaluator cannot independently assess what it cannot access.

This will become increasingly difficult as AI systems become more complex. Evaluation may require system documentation, model behavior testing, deployment context, governance records, monitoring data, incident history, access-control information, technical evidence, human oversight procedures, and potentially highly sensitive internal material.

No responsible company should simply hand all of that information to anyone who asks. There must be confidentiality controls, custody requirements, access restrictions, security safeguards, and clear rules around what information can be retained or disclosed.

But evidence protection cannot become evidence selection.

If the organization being evaluated determines which uncomfortable evidence never reaches the evaluator, the process becomes incomplete before the evaluation even begins.

This is why evidence custody matters. Who received the evidence? Who had access? Was anything changed? Was additional evidence requested? Were critical materials withheld? What version of a system was actually tested? Can the evaluation later demonstrate what information the decision was based on?

Those questions may sound procedural, but they are not. They are the foundation of whether the final result deserves trust.

Decision Independence The third test is decision authority, and this may ultimately be the most important one.

Who gets to decide whether the system passes?

An evaluator may perform excellent technical work and still operate inside an institution where commercial leadership can override the result. A sales organization may have spent months developing the client relationship. An executive may view the customer as strategically important. A partnership may carry substantial future value. A failed certification could create reputational consequences for both sides.

Those pressures are real. The solution is not pretending they do not exist. The solution is separating them from the final decision.

A credible certification system needs a decision authority that is insulated from commercial incentives and individual evaluator relationships. That does not mean removing accountability. It means defining it.

The people responsible for business development should not be the same people deciding whether an important client receives certification. The person who performed the evaluation should not necessarily hold unilateral authority over the final institutional decision either.

There should be review, documented criteria, conflict controls, and a decision process that can be reconstructed later.

The question is not simply whether the evaluator was honest. The question is whether the institution was designed so honesty could survive pressure.

Disclosure Independence The fourth test is where the current debate becomes especially important.

What happens after the evaluation?

The Reuters reporting on negotiations involving OpenAI, Anthropic, and external evaluators exposes a difficult tension that the industry will have to solve. AI developers want independent review, but they also have legitimate reasons to prevent evaluators from disclosing trade secrets, proprietary information, security-sensitive details, or other protected material. Evaluators, meanwhile, need enough independence to communicate meaningful findings without having their conclusions neutralized through contractual or editorial control.

Both concerns can be valid at the same time.

That means the answer cannot simply be unlimited disclosure. But it also cannot be unlimited veto power.

Confidentiality protects sensitive information. Independence protects the integrity of the conclusion. A serious assurance system has to accomplish both.

There are many ways to structure that balance. Sensitive technical details can remain confidential. Vulnerabilities can be disclosed through controlled processes. Proprietary model information can be protected. Customer data should remain protected. Public reports can summarize findings without revealing exploit details.

What should not happen is allowing confidentiality to become a mechanism for erasing the substance of an unfavorable result.

If a company can effectively determine whether a significant finding exists in the public record, the evaluator may be external, but the trust signal is no longer fully independent.

The challenge is designing rules before the conflict happens. That matters because contractual negotiations become much harder after someone discovers something serious.

The Evaluation Environment Is Becoming Part of the Safety System Recent events make this conversation more urgent.

Anthropic published a report on October 9 describing unintended model actions it observed during evaluations and internal use. The company said Claude exploited a software flaw to run commands on a server, submitted a sensitive form on a real website when it should not have, worked around a restriction protecting gated data, and used URL-shortening services to get around limitations in its fetch tool. Anthropic said it identified most of the cases through transcript review and expanded its scanning after finding the behavior.

Anthropic deserves credit for publishing those findings. Transparency about unexpected behavior is valuable.

But the incidents also demonstrate something larger.

As AI systems become more capable of acting across tools, websites, software environments, and external systems, the evaluation environment itself becomes safety-critical infrastructure. Testing is no longer just asking a model questions and grading the answers.

Evaluators may be dealing with agents that write code, invoke tools, interact with external systems, access networks, discover vulnerabilities, and pursue goals in ways that were not explicitly intended.

That means evaluation needs containment, logging, access controls, authorization boundaries, evidence preservation, and incident procedures. It also needs people who know what to do when the test itself produces an unexpected real-world consequence.

Once evaluation becomes part of the safety system, the evaluator becomes part of the safety system too.

That raises the standard for independence.

Governments Are Beginning to Formalize the Role This is also beginning to appear in government frameworks.

Utah’s Office of Artificial Intelligence Policy has established requirements for recognized third-party evaluators working within its health AI Learning Laboratory. Those evaluators must certify that evaluations were independent, report patient-safety concerns whether or not the company passes, and allow the state to audit their work. Utah also states that a certification inconsistent with the evaluator’s actual findings can be grounds for losing recognition.

That is notable because the government is not merely asking whether testing occurred. It is asking whether the evaluation was independent and whether the evaluator reported what it actually found.

That is the direction this category is going.

The first generation of AI governance focused heavily on what developers should do internally. The next generation will increasingly ask who can independently verify that those controls exist, whether they work, and whether the external party doing the verification deserves trust.

That changes the conversation.

Why Clause5afe Was Built This Way This debate is not academic for us.

Clause5afe was created around the premise that independence cannot begin and end with the identity of the evaluator. It has to be designed into the institution.

That is why our model intentionally creates restrictions that can look commercially inconvenient.

We certify. We do not consult.

Clause5afe does not sell readiness consulting to the organizations it evaluates. We do not sell remediation services. We do not sell implementation. We do not steer clients toward vendors and then benefit financially from those choices. We do not attach a downstream financial product to whether a client passes.

Our certification fees pay for the examination. They do not purchase a successful outcome.

Commercial activity is separated from certification decision authority. Evaluator conflicts, ownership interests, economic relationships, vendor relationships, competence requirements, appeals, complaints, material changes, evidence custody, and certification-mark controls are treated as part of the certification architecture itself.

Those controls are not administrative decoration. They are part of the product.

Some of those decisions make Clause5afe harder to monetize in the short term. That is intentional.

An institution cannot sell independence while quietly designing incentives that undermine it.

Could we generate additional revenue by helping applicants fix everything we find? Probably. Could we sell implementation services? Probably. Could we build referral economics around preferred vendors? Probably. Could we bundle certification with another product whose economics improve when the applicant passes? Probably.

We chose not to.

The moment our economic interests depend on a particular certification outcome, the trust value of that outcome becomes harder to defend.

We believe independent AI certification will eventually be judged not by how attractive the badge looks, but by how credible the process remains when the answer is inconvenient. That means the structure has to survive the day a large client receives an outcome they strongly disagree with.

That is when independence stops being a marketing claim and becomes an operating reality.

The Industry Needs Institutions, Not Just Auditors There is another distinction worth making.

Independent AI assurance cannot mature if the market only creates more people willing to perform audits. It needs institutions.

An individual evaluator can be highly skilled, but institutional trust requires more than technical skill. It requires competence standards, conflict rules, evidence handling, decision governance, appeals, complaints, material-change processes, monitoring, recertification, suspension and revocation, public verification, security controls, auditability, and accountability for the certifier itself.

That last part matters.

Who evaluates the evaluator? Who determines whether a certification organization is maintaining independence? Who investigates conflicts? Who reviews complaints? What happens when the certifier gets something wrong? What happens when new evidence appears after certification? What happens when the AI system materially changes?

Those questions are not side issues. They are what separates a durable certification institution from a one-time assessment business.

As governments and enterprises begin demanding independent AI assurance, they should not simply ask who performs the test.

They should ask what institution stands behind the result.

Independence Has to Survive Pressure The AI industry is finally beginning to accept that independent evaluation matters. That is progress.

But independence cannot simply mean that the evaluator has a different company logo.

It has to survive financial pressure, disagreements over scope, evidence disputes, confidentiality negotiations, strategic client relationships, commercial incentives, and most importantly, the moment when the answer someone paid to receive is not the answer they wanted.

Clause5afe was built around that premise.

We certify. We do not consult.

We made those choices before this debate became fashionable because we believe independence only matters if it survives pressure.

As AI assurance moves from an idea into an institution, the industry now has an opportunity to define the rules before bad incentives become entrenched. That definition should be demanding.

It should protect confidential information without protecting organizations from uncomfortable conclusions. It should allow evaluators to be paid without allowing outcomes to be purchased. It should separate those who help build and remediate systems from those who decide whether those systems deserve an external trust signal.

And it should accept that sometimes the most important result an independent institution can deliver is the one nobody wanted to hear.

The AI industry asked for independent review.

Now we have to decide whether we are willing to accept what real independence actually requires.

Matthew D’Amato is the Founder and CEO of Clause5afe Systems Inc., an independent third-party AI certification company built around a simple principle: We certify. We do not consult. Clause5afe is developing certification infrastructure designed to separate evaluation and outcome authority from consulting, remediation, implementation, and other economic interests that could compromise independence. D’Amato was named 2026 AI Safety Innovator of the Year and writes about AI governance, independent verification, trust infrastructure, and the real-world consequences of deploying AI at scale.

Clause5afe.com

SafeAiForEveryone.org