AI needs a 'brake pedal', warns Anthropic co-founder

Anthropic's widely used chatbot Claude now operates on code that is 80 percent self-written, the company's co-founder said, with full automation possible within two years^[1][2]. The AI firm is simultaneously preparing for a public stock market debut with an estimated valuation approaching $1tn^[1][2].

Jack Clark, co-founder of Anthropic, told BBC Newsnight that reaching 100 percent self-written code for its systems "would have huge implications"^[1][2]. He used the milestone to advocate for greater regulatory oversight, stating the AI industry currently has "a gas pedal, but it doesn't have a brake pedal"^[1][2]. Clark argued that society needs to develop a policy framework to maintain human control over increasingly powerful systems^[1][2]. Despite these public warnings, major AI companies, including Anthropic, OpenAI, and Google, have not announced any pause in their research efforts^[2]. Anthropic's rapid growth since its founding five years ago has led to its anticipated public listing, which is poised to be one of the most valuable in history^[1][2]. The company has positioned itself as outspoken on AI risks, even clashing with the U.S. Department of Defense over concerns about surveillance and autonomous warfare applications^[1][2]. Clark stated the motivation for discussing AI capabilities is to inform the public about developments inside these companies, not to burnish its commercial reputation^[1][2].