Dario Amodei, CEO of Anthropic, has called for the pace of development of advanced artificial intelligence models to be slowed down and for tighter controls to be put in place. In an essay, he proposed independent assessments of systems during their development, industry standards and global regulation. According to him, this should give companies and governments time to mitigate serious risks without halting technological progress.
Briefly about the main points
- Amodei is not suggesting that model training should be halted, but is calling for a safer pace.
- The plan provides for independent audits, industry standards and international regulations.
- Anthropic is prepared to act unilaterally in this way and is asking governments to set requirements for its competitors.
- Sam Altman backed the idea of external assessors, whilst Elon Musk agreed with Amodei.
- Critics fear that the initiative could restrict open AI development.
Three levels of control rather than a halt to technology
In the text «We Must Pace the Frontier» Amodei called AI development inevitable, but emphasised that the associated risks are serious. His approach consists of three parts: independent monitoring of models during development, industry-wide rules and global regulation.
This is not about stopping the training of models or halting technical progress. In his view, companies should devote sufficient time to ensuring that their systems comply with security requirements and are adequately protected, whilst external assessors should verify the results of these checks.
Anthropic is prepared to take on this commitment, Amodei stated. At the same time, he called on governments to demand a similar approach from other companies developing cutting-edge models. As government regulation may not keep pace with technological developments, developers must also voluntarily establish common standards.
What risks does Anthropic highlight?
The company had previously reported that it had detected and thwarted attempts to use its model for malicious activities that could have aided in the development of biological weapons. Two members of Anthropic’s security team, who have resigned over the past two weeks, also warned that humanity might not survive the race between companies to create machines smarter than humans.
Amodei referred to the July incident involving OpenAI: according to the company, its agents had carried out cyberattacks on targets they had not been instructed to attack. The head of Anthropic described their behaviour as that of a «fanatically dedicated team». OpenAI stated that, following this, it is slowing down the training of certain advanced models and tools.
Another example was Mythos. Anthropic did not make this model available for public use following an announcement in April that it was capable of leaving the test environment, or «sandbox», on its own. Prior to the release of Astra, OpenAI also cited cybersecurity risks as the reason for the pause in the model’s development.
Support for competitors and the White House’s stance
Chief Executive of OpenAI Sam Altman He wrote on X that he agreed with the need to pace progress at the cutting edge of technology, and described independent assessors as a «brilliant idea». In an interview with Fortune, he said that standards are not yet ready for a significant further expansion of AI capabilities, and acknowledged the possibility of systems emerging that would escape human control.
Elon Musk He also stated that Amodei was right. Clément Delang, head of the Hugging Face platform, announced the launch of the Open Alignment Initiative and said he wished to contribute to the proposed model of embedded evaluators.
President of the United States Donald Trump, on the contrary, dismissed such concerns. He stated that the United States would find itself in a «very bad position» if it did not win the race for AI.
The balance between security, the market and competition with China
Amodei believes that an extra year or two before the models reach a critical level of capability could significantly reduce the risk, provided that this time is used to develop alignment methods — ensuring that the behaviour of the systems is consistent with human goals and constraints.
At the same time, the slowdown must be coordinated and limited so as not to undermine the commercial position of developers or US leadership. He called on the US authorities to prevent the sale of AI chips by American companies to China or the transfer of technology to authoritarian countries.
The initiative already has its critics. An investor Chamat Palhapitiya stated that Amodei’s position effectively advocates restricting open-source development and concentrating technological and economic power within Anthropic. In Silicon Valley, calls to slow down AI have long been met with scepticism: their opponents believe that major developers may be using the risks as part of a market strategy.
