A leading British politician has urged the United Nations to intervene in “the unsafe development of super intelligence” after an Anthropic team leader posted: “We really do earnestly believe AI could kill all humans!”
Whether “marketing hype” or signposting of a genuine extinction threat, “governments must step in,” Labour MP Darren Jones wrote in an open letter also addressed to the UK prime minister and international Organization for Economic Cooperation and Development.
Jones is the latest lawmaker scrambling to find ways to protect the British public from AI threats to safety. Just one day earlier, Labour MP Alex Sobel had introduced a parliamentary bill proposing to ban the “development, deployment, and operation of artificial superintelligence”—systems which can fully replace and outmatch humans at any task. The legislation would also give the government powers to monitor and control the development of the technology at different points in the supply chain, including at chip level.
“We are not alone in this effort,” said Sobel while introducing the bill, pointing to the US government’s temporary export ban on Anthropic’s Fable 5 and Mythos 5 models in June. “If the Trump Administration, who are perceived as historically against AI regulation, can contemplate drawing a line on models that are too powerful and dangerous, it would be strange for us not to ask what our response should be,” he argued.
MPs from across political parties put their names on the bill, including Labour MP Jess Asato, who is suing Elon Musk’s xAI after its Grok chatbot was allegedly used to generate fake sexualized images of her. Lawmakers have become increasingly worried about the threat of AI to national security after a string of incidents of AI agents going rogue in recent months. They want to introduce guardrails against worst-case scenarios, such as these systems seizing control of critical infrastructure and running sophisticated influence operations.
The Hugging Face hack by OpenAI’s models that escaped containment in July “is why it’s even more important that we move quickly to regulate and legislate on superintelligent AI,” Sobel tells WIRED, “because AI is developing at pace, and I’m afraid to say those developing it don’t know exactly what’s in it.”
In an unusual move, the parliamentary bill was drafted by the nonprofit ControlAI, which is backed by Skype founding engineer and former DeepMind investor director Jaan Tallinn. He had previously been working at AI safety startup Conjecture but tells WIRED he wanted to make the public and policymakers aware of the “open secret” that “developing superintelligence could cause human extinction.”
Conjecture founder Connor Leahy now leads ControlAI’s US arm, which consulted on Bernie Sanders and Greg Casar’s bill to ban superintelligence development, in addition to having briefed nearly 200 congressional offices and more than 20 members of the Senate and House of Representatives on the extinction risk of superintelligence.
Sobel will present his bill in full in November. While he accepts it’s rare for the UK government to accept private legislation like this, Sobel hopes it may lead the government to work with other countries on an international ban.
Sobel has previously tried in vain to introduce a “kill switch” into the Cyber Security and Resilience Bill, which would allow the government to pull the plug on any data center hosting an AI agent that presented a threat. The mechanism, which was also proposed by ControlAI, is similar to a law pushed for by both Republican and Democratic lawmakers in the US. AI minister Kanishka Narayan, then working under former prime minister Keir Starmer, shot it down, pointing out that the bill already allows the government powers to intervene if compromised network and information systems cause a national security risk. That could include requiring an entity to cease using and isolate an AI model, he noted.
But that may not be enough, Sobel says, hoping to meet with the government to hammer out details of which companies would be included within the scope of the proposed rules. “I think this might be the crucial year,” he says. “If we leave it too long, things may move past our ability to regulate it.”
Peers across parties in the UK parliament’s upper house, including former AI minister Viscount Camrose, tried unsuccessfully to reintroduce the measure last week. They have been frustrated as the binding regulation on frontier AI firms promised by the Labour government when they won power in 2024 has failed to materialize.
While peer Tim Clement-Jones admitted they shouldn’t be trying to “shoehorn” AI regulation into a cybersecurity bill. In the absence of any other AI legislation, “we have no choice,” he said in a speech to parliament. “We must have more control and oversight over that sort of behavior, because you can see its consequences,” said Alexandra Freeman in the debate, referring to the Hugging Face incident. “This legislation is our only opportunity now.”
Nearly nine out of 10 Britons support independent regulation of the technology, a poll by the Ada Lovelace Institute showed in December. Meanwhile, nearly three times as many (38 percent) believe that the risks of AI outweigh the benefits as vice versa (13 percent), according to official statistics. But “the government has been completely deaf to this, and they have paid no attention to those who are advocating for this,” Clement-Jones says, arguing that the public’s lack of trust in the technology will curb adoption.
The UK government evaluates the capabilities and safety of advanced artificial intelligence models through the research body the AI Security Institute. Cooperation is voluntary for AI companies, as the organization has no legal powers to access unreleased models for testing or ban their public release. For example, the agency reportedly did not get access to Anthropic’s latest model, Claude Mythos 5.1, for testing.
“I think it’s a rather naive position to imagine that we can do things on a voluntary basis,” says Clement-Jones. “Do you really think that the big AI companies have got everybody’s welfare at heart?”
