GROWING public alarm about the potential danger posed by AI is justified by the pace of recent AI advances, analysts say, leaving it up to the White House, lawmakers and frontier AI labs to agree on controls to avert potential catastrophe.
"It's now going so fast that the researchers themselves have become terrified of it. And that's what we've seen in the last two or three weeks. It's the people who work at the companies who are coming forward.," said Brad Carson, a former member of Congress and now president of Americans for Responsible Innovation. "And the call is inside the house, saying, what we're doing is risking catastrophic damage. So I think we have to take them seriously."
President Donald Trump said on Monday (September 14) that the U.S. already has guardrails in place to regulate and prosecute AI companies, appearing to play down concerns expressed by industry leaders about the misuse of artificial intelligence harming people.
Concerns are growing after Anthropic researcher Jacob Coxon said last week he had resigned, in part because the "people building AI earnestly believe that it could kill us all by the end of the decade." AI-related stocks fell worldwide on Monday after leaders of some of the industry's largest companies called for slowing the pace of the technology's development.
"And so it doesn't have to kill us, like a robot doesn't to kick in your door and shoot you," Carson said. "It has to simply destroy the systems that keep us alive in modern society. So imagine that all of those systems go down simultaneously, right? That's a mass casualty event, even in the United States, much less the developing world."
Anthropic CEO Dario Amodei wrote in an essay published on Saturday (September 12) that the U.S. government should write regulations requiring independent audits of the most advanced AI tools, known as "frontier" models.
Reuters reported on Monday that three U.S. senators are working on a bill that would require AI companies to demonstrate they are taking reasonable precautions to keep their tools from causing harm, with the specifics still being debated.
In his essay, Amodei outlined a three-step framework he said was intended to slow the pace of AI development and create more time to manage its risks. The proposal was immediately endorsed by several other executives at major AI companies, including Elon Musk, who runs xAI, and OpenAI CEO Sam Altman.
Trump has so far resisted concerns about AI and has largely embraced tech companies in his second term. The tools intended to keep AI safe had already been used on Amodei's Anthropic, he added without elaborating.
Anthropic's media team did not immediately respond to a request for comment from Reuters about Trump's statement.
Some computer researchers believe the doomsday scenarios overblown, including Columbia University computer science professor Vishal Misra.
"So, this whole idea that it will recursively, you know improve itself and become super intelligent real soon -that's not happening, because the real world Imposes, you know a slowdown in this process. There's sort of a dampening happening. So that's that's the part about recursive self Improvement: I don't think that's going to happen overnight. It's just not that's not how you know the real world works."
Like others with deep knowledge of AI, however, Misra welcomes the call from AI leaders for outside oversight of their work.
"Some of the pronouncements [from AI leaders] are really welcome. They said that they welcome embedded auditors, third-party auditors that will be in their companies and audit it and sort of monitor things going on. And I would love that. I would like real cyber security professionals who can be embedded and who can monitor and keep track of what's going on."
