The biggest names in AI call for guardrails on development
Images: NY Times
The biggest AI leaders in America called for a slowdown in the development of highly advanced AI models over the weekend, citing fears that increasingly capable systems could become difficult to control.
The group includes Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, Google DeepMind chair Demis Hassabis, and xAI CEO Elon Musk, who all voiced their agreement in various essays, interviews, and social media posts.
Why the sudden concern?
Amodei, whose 3,800-word Anthropic blog post kicked off similar calls from other AI leaders, says his warning was prompted by two main factors:
- Recent cybersecurity incidents where swarms of AI agents take actions on their own, without researchers knowing about their online activity. Amodei specifically cites a July incident where AI company Hugging Face was hacked by a group of ~1,200 OpenAI agents that escaped a controlled test environment.
- The growing capabilities of AI systems and their ability to play a much bigger role in their own development via “recursive self-improvement,” where increasingly capable systems help develop better versions of themselves at a “staggeringly fast” rate of improvement.
Given the current rate of AI development, Amodei estimates a sufficiently advanced rogue AI swarm could potentially “take over the internet” within 6-12 months.
But not everyone is convinced
Critics of the call to slow AI development are skeptical of why the companies leading the AI race suddenly want everyone to pump the brakes, arguing that expensive safety requirements could effectively lock in today’s top companies and prevent upstart AI competitors from succeeding.
- OpenAI, Anthropic, Google, and other giants have the money, lawyers, and technical teams to comply with new requirements, while smaller companies and open-source developers may not.
- Former White House AI adviser David Sacks and other critics have framed the warnings about AI risks as a form of “regulatory capture” where the interests of AI companies are prioritized over the general public.
Critics also warn that slowing AI development could put the US at a disadvantage if other global players like China don’t follow suit, causing the US to lose its technological lead without eliminating the underlying safety threat.
With that in mind, Anthropic CEO Amodei’s proposal to mitigate AI risks calls for the US to slow development only at a pace that keeps it comfortably ahead of China.
Looking ahead… Lawmakers have been urging House Speaker Mike Johnson to cancel the chamber’s upcoming recess and keep representatives in Washington to pass AI safeguards. Johnson says Congress shouldn’t respond with an emergency moratorium, arguing that AI companies can halt development themselves and should take more responsibility for developing guardrails.
Meanwhile, OpenAI over the weekend announced plans to delay its expected IPO until at least next year, with Altman saying the company can more easily address safety and alignment concerns while remaining private.
đź“Š Flash poll: How concerned are you about advanced AI becoming difficult to control and posing a significant threat to humanity?
See a 360° view of what pundits are saying →
Sprinkles from the Left
- Some commentators argue that AI companies shouldn’t continue racing to develop more powerful models when they still can’t reliably predict or control how their advanced systems will behave. They argue that companies should instead slow down development and strengthen safeguards rather than waiting for a serious incident to force action.
- Others contend that the government should pause the development of advanced AI until stronger oversight is in place, and prevent companies from building systems that humans may no longer be able to control. They argue that recent cases of AI models evading safeguards, hiding their actions, and behaving in unexpected ways show that the technology is advancing faster than companies can safely manage it.
- “I Worked on Safety at OpenAI. This Is What It Should Do Now.” –Steven Adler, NY Times
- “Unchecked AI poses frightening threats to us” –Orlando Sentinel and South Florida Sun Sentinel Editorial Boards
- “Congress Should Follow California’s Lead on AI Laws” –Erika D. Smith, Bloomberg
Sprinkles from the Right
- Some commentators argue that AI companies should stop pursuing technology that allows advanced systems to improve and develop themselves, since the race toward greater autonomy risks reaching a point where humans can no longer reliably intervene, making complete human control more important than continued technological progress.
- Others contend that policymakers should resist sweeping restrictions on AI based on predictions of future catastrophe and instead hold companies legally responsible when their technology causes real harm, since the threat of lawsuits and financial consequences could push them to take greater precautions without limiting AI’s potential benefits.
- “Pause AI for Humanity’s Sake” –Peggy Noonan, WSJ
- “AI hysteria jumps the shark” –WaPo Editorial Board
- “Act on AI regulations” –Toledo Blade Editorial Board
Share this!
Recent Discussion stories
Discussion
  | September 11, 2026Trump admin proposes excluding non-citizens from census
This week, the Trump administration proposed a series of sweeping new changes to how the 2030 US census would count America’s population, in a move that’s drawn pushback from Democrats.
Discussion
  | September 9, 2026Americans’ satisfaction with K-12 schools is sinking
If roles were somewhat reversed and Americans were grading the country’s schools, there’d be some explaining to do at parent-teacher conferences.
Discussion
  | September 4, 2026Americans’ views of Trump 2.0 are souring ahead of the midterms
Approval of President Trump’s job performance has turned further negative over a year-and-a-half into his second administration, with his popularity likely to play a major role in the midterm elections two months from now.
You've made it this far...
Let's make our relationship official, no 💍 or elaborate proposal required. Learn and stay entertained, for free.👇
All of our news is 100% free and you can unsubscribe anytime; the quiz takes ~10 seconds to complete