Anthropic has urged major AI labs to slow the pace of AI development, the Wall Street Journal reported on Wednesday.

In a blog post, Anthropic warned AI systems are approaching a stage of “recursive self-improvement,” in which they improve themselves without human intervention. It also said slowing AI development would be “beneficial to the world.”

The post was written by Marina Favaro, head of an internal research institution at Anthropic, and co-founder Jack Clark.

They wrote that recursive self-improvement has not yet been achieved and is not inevitable, but “could arrive before most organisations are prepared.” They also proposed a global agreement to slow AI development and a system to verify compliance. Comparing it to nuclear weapons treaties, they said that unlike missile bases, there is no way to verify AI training from the outside, making it far harder to catch those who break agreements.

Clark also said in a lecture in London last month that recursive self-improvement “could happen within 2 years, perhaps sooner.” Anthropic plans to hold related discussions with policymakers and researchers in the coming months.

Some say Anthropic’s warnings on AI safety are aimed at checking competitors or marketing. Venture investor David Sacks criticised it, saying Anthropic executives are pushing “regulatory capture,” in which companies wield influence over governments or policymakers to shape regulation in their favour.