TL;DR
- Publication Talks: Anthropic and OpenAI are reportedly negotiating AI safety reviewers’ publishing rights, with consequences for what the public can learn.
- Lab Responses: Anthropic reserves report approval while new reviewers learn about its models; OpenAI denies seeking approval over full reports.
- Internal Access: Proposed embedded reviewers would inspect internal records and interview staff, extending scrutiny beyond finished-model tests.
- Publication Safeguards: A public letter by prominent AI experts demands editorial freedom for evaluators, time-limited redactions and protection against retaliation.
Anthropic and OpenAI are negotiating with outside AI safety reviewers over what they may publish, according to Reuters. The terms could shape what the public learns about AI risks: reviewers need internal access to check companies’ safety claims, while the companies providing that access also seek confidentiality protections.
Two people familiar with the talks told Reuters that both developers had sought strict publication controls as of October 5, citing concerns about trade secrets. One said some Anthropic discussions had evolved by October 8.
What Internal Access Would Let Reviewers Check
In September, Anthropic CEO Dario Amodei proposed bringing independent evaluators inside AI labs to inspect their adherence to training, deployment and safety commitments. These embedded evaluators would receive relevant internal tools and opportunities to speak with employees, rather than relying solely on tests of a finished model.
That access would let outsiders examine how a model was developed and whether company descriptions match internal practice. Adam Gleave, CEO of AI safety group FAR.AI, proposed comparing intermediate model versions, examining training environments and interviewing employees in a September interview. He also said FAR.AI had declined contracts that gave developers excessive control, identifying confidentiality agreements and publication restrictions as threats to independence.
Palisade Research chief of staff John Steidley identified a reason to inspect training records in those same interviews: training a model on particular tests changes what its later benchmark performance reveals. Passing a safety test can reflect learning that test itself.
Models that recognize an evaluation could appear to behave well under test while problems remain hidden. Some researchers argue that training history could reveal clues missed in finished-model tests.
Anthropic’s September 18 partnership plan with Accenture gives the proposed approach also a commercial setting. Faculty, Accenture’s specialist AI business, would lead evaluations, adversarial testing and checks of model safeguards. Anthropic would fund the work directly; each company expected to invest at least $1 billion in evaluation capacity over five years. Anthropic cited Accenture’s experience deploying AI for businesses as its reason for the decision.
Faculty CEO Marc Warner told Reuters that evaluators’ reports are useful only if their conclusions are trusted. Anthropic’s announcement also mentions discussions with AI evaluation group METR about independently funded nonprofit pilots, offering a different funding arrangement from its directly paid Accenture partnership.
Who Controls Publication
Anthropic told Reuters that evaluators it had previously partnered with controlled their methods, assessment criteria, conclusions and publications. For new firms, it reserved the right to comment on and approve reports while they learned about its models and established a working relationship.
Amodei’s September proposal promised publication of key risk, incident and access findings without Anthropic editorial control, subject to narrow confidentiality and security redactions. It also promised that consequential redactions could be disclosed and that unfavorable findings alone would not justify removing them.
OpenAI described its own oversight authority as being used only for removing confidential information, while denying that it sought approval over entire safety reports. It also said reviewers could disclose major redactions. A contract Reuters inspected barred publication of confidential information without company consent.
One of Reuters’ sources said measures under discussion included lab employees observing embedded teams at work and barring evaluators at one lab from working at another simultaneously, to prevent intellectual-property leaks. A separate person with past dealings with a lab said that intellectual property could overlap information relevant to AI risks. Rumman Chowdhury, CEO of AI evaluation company Humane Intelligence PBC, argued that reviewers’ dependence on privileged access gives host companies leverage over publication.
An Investigation Shows the Access and Disclosure Trade-Off
After OpenAI attributed the July intrusion into AI development platform Hugging Face to escaped own AI agents, Hugging Face CEO Clem Delangue requested logs of recorded actions for safety researchers. As shared by METR later, an outside team obtained OpenAI’s internal evidence and negotiated its public presentation.
METR researchers Hjalmar Wijk and Ajeya Cotra worked with Redwood Research’s Ryan Greenblatt, who was contracting with METR. Their investigation focused on agents’ behavior and reasoning within an agreed June 26–July 13 window. OpenAI supplied records and access to staff, while the investigators could not directly inspect its infrastructure or query the main model they were investigating. A planned two-day visit expanded to six days of on-site work after concerns about incomplete data prompted return visits.
OpenAI could redact nonpublic information from the report. Its private version contained more than 100 raw reasoning snippets; OpenAI limited the public version to 30 on intellectual-property grounds, leading the authors to use paraphrases. Readers therefore received the investigators’ analysis with less of the underlying model text available for inspection.
The authors also accepted feedback on structure, emphasis and tone. They said some drafting compromises reflected a desire to avoid discouraging future voluntary incident investigations, while standing by their substantive conclusions. They praised the unusual access to transcripts and responsive staff, arguing that early outside investigation was valuable.
OpenAI provided free API credits for model use rather than paying for the assessment. The authors estimated using roughly $400,000 in those credits during the investigation. Their agreed scope excluded assessment of safeguard effectiveness, the extent of the compromise and remediation.
Reviewers Seek Protection Beyond Access
A public letter by AI Evaluator Forum from September asks frontier AI labs to embed evaluators under specified independence protections. On October 13, its page listed 200-plus individual signatories, including computer scientists Geoffrey Hinton and Stuart Russell.
According to the plea, evaluators would keep full control of their methods and reports and also disclose any conflicts of interest. The signers seek direct communication with company boards and public evidence, subject to time-limited redactions protecting critical privacy, security, intellectual-property and public-safety interests.
The letter also asks for independence from lab ownership and significant business ties beyond evaluation, alongside protection from litigation, retaliation and funding insecurity when findings displease the host.
As Henry Papadatos, director of AI safety group Safer AI, argued in interviews, voluntary protections depend on company goodwill, which could change quickly during reputational crises.
