Anthropic calls for a global slowdown in AI development over fears systems could build successors
Anthropic urges a pause
Anthropic is warning that artificial intelligence is advancing toward a point where systems may be able to help build their own successors, and it says that possibility should prompt a global slowdown in AI development.
In a blog post, the company said that while AI capable of developing successor systems could deliver major benefits in science, healthcare and other fields, it could also raise the risk of humans losing control over increasingly capable models. Anthropic argued that a temporary pause or slowdown would give society and AI safety research time to catch up.
The company said the trend is moving quickly enough that this milestone “could come sooner than most institutions are prepared for,” even though it is not here yet.
Verification would be the hard part
Anthropic said any slowdown would only work if major AI labs around the world agreed to stop or reduce development under the same conditions. It also said there would need to be reliable ways to verify that competitors had actually paused, warning that otherwise some organizations could continue advancing in secret.
“A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions,” the company wrote. “It would also require that each can verify that the others have actually stopped.”
The company’s proposal is grounded in work from Anthropic Institute, a research division it launched in March to study the challenges that emerge as AI systems become more advanced.
Skepticism follows the warning
Protect your privacy with Doppler VPN
3-day free trial. No registration. No logs.
Anthropic’s call has also drawn skepticism. The Wall Street Journal noted that critics see the company’s public warnings as a possible marketing strategy, aimed at making Anthropic appear more responsible than rivals or suggesting its own products are the safest option.
Those critics have pointed to the limited release of Anthropic’s cybersecurity model Mythos, which the company said was restricted to select partners because of the potential harm its vulnerability-finding abilities could cause in the wrong hands.
Anthropic, meanwhile, is one of the most prominent names in the AI race and is reportedly on track for its first profitable quarter. It recently filed paperwork with the SEC to go public, likely before the end of the year.
Sources:
Read more tech news on the Doppler VPN Blog.