Anthropic calls for ‘brake pedal’ on AI development

Anthropic, the world’s most valuable artificial intelligence start-up with a $965 billion valuation, has called for a global “brake pedal” on AI development, warning that humanity is rapidly running out of time to retain control of the technology.

The company behind the Claude chatbot offered to suspend its own work on advanced frontier systems, warning that the industry currently possesses a ‘gas pedal’ with no mechanism to slow down.

The primary driver for this urgent appeal is the imminent threat of “recursive self-improvement,” a threshold where AI systems begin to autonomously design, code and advance themselves without human intervention.

Anthropic executives revealed that 80 per cent of the code operating their current systems is already written by the AI itself, with staff producing eight times as much code as they did between 2021 and 2025.

Company president Jack Clark warned that the technology could achieve 100 per cent autonomous development within two years, a milestone that would trigger an explosion in capabilities that humans may be fundamentally unable to manage or control.

The core problem spans catastrophic digital risks and severe macroeconomic disruptions. On a technical level, Anthropic has already withheld its most powerful model, Mythos, from public release due to fears it could be weaponised to execute devastating cyberattacks.

Chief executive Dario Amodei previously estimated a 25 per cent chance that the technology’s trajectory goes “really, really badly.” On a societal level, the company warned of major economic fallout as autonomous AI “agents” begin overtaking routine employment, noting that recent mass tech layoffs are already tied to software engineering tasks being offloaded to AI.

However, Anthropic admitted that co-ordinating a global freeze resembles a complex international arms control problem. A meaningful pause would require fierce geopolitical rivals like the US and China, alongside multiple leading labs, to agree to identical terms.

Furthermore, verification is nearly impossible because massive AI training runs are far easier to conceal than traditional military hardware like missile silos.

If a unilateral slowdown simply allows less cautious actors to catch up, executives warned it could leave the world even more vulnerable.

While Anthropic urged governments to build a robust regulatory framework mirroring how society reined in the oil barons of the last century, critics contend the company is inflating threats to encourage regulations that would block rivals from entering the market.