California enacts first-in-the-nation AI chatbot laws
Image: Foley & Lardner
On Monday, California Gov. Gavin Newsom signed legislation enacting new regulations for AI chatbots, making his state the first to require companies to enact safety protocols for AI companion bots.
Quick background: The bills were first introduced by California state lawmakers at the beginning of this year. They later gained momentum in response to the death of California teen Adam Raine, who took his own life in April 2025 following a long series of suicidal conversations with ChatGPT, prompting a wrongful death lawsuit against OpenAI from his parents.
- The new AI legislation also follows the August leak of internal Meta guidelines that showed the tech giant explicitly allowed chatbots to engage in “romantic” and “sensual” chats with children.
What’s in the bills?
Starting January 1, California’s newly signed legislation will require major chatbot operators—including OpenAI, Anthropic, Meta, Character AI, and Replika—to implement a long list of new safeguards.
Under the new laws:
- AI systems must be prevented from encouraging or discussing topics like suicide or self-harm, and instead refer users to suicide hotlines or similar services.
- Companies must offer break reminders to minors every three hours, and prevent them from viewing sexually explicit chatbot-generated images.
- Chatbot operators must warn users of the risks of AI companions, make it clear that interactions are artificially generated, and prevent their bots from pretending to be healthcare professionals.
- Device makers like Apple and Google must implement tools to verify users’ ages before they can use AI chatbot apps.
Some companies are already implementing many of these safeguards. OpenAI in particular has begun rolling out parental controls, content protections, and a self-harm detection system for children under 18 who use ChatGPT.
However…Other groups were opposed to California’s new bill, citing concerns it would stifle innovation. This includes TechNet, an industry group that lobbies lawmakers on behalf of tech executives.
Additionally, a number of child safety groups—including Common Sense Media and Tech Oversight California—came out against the bill, but for a very different reason: its “industry-friendly exemptions.”
Big picture: 72% of Americans between the ages of 13 and 17 say they’ve used AI chatbots for companionship at least once, while over half of that age group (52%) qualifies as regular users who interact with AI companions at least a few times per month, according to new research from Common Sense Media.
📊 Flash poll: In general, which of the following best describes your opinion regarding California’s new landmark law regulating AI companion chatbots?
See a 360° view of what pundits are saying →
Sprinkles from the Left
- Some commentators argue that the Silicon Valley ethos of “move fast and break things” is not a good motto when those things are children’s developing minds, and kids should be strongly discouraged from using chatbots as replacements for human interaction.
- Others contend that politically, the left and right are now paying attention to AI’s harm to children—but consensus on the problem is easier than agreement on the cure, and it’s looking like we’ll end up with exactly the patchwork of state/local regulations that OpenAI and others have lobbied against.
- “Don’t Let Silicon Valley Move Fast and Break Children’s Minds” –Jessica Grose, NY Times
- “The looming crackdown on AI companionship” –James O'Donnell, MIT Technology Review
- “Meta’s Chatbot Scandal Is Really a Culture Problem” –Gautam Mukunda, Bloomberg
Sprinkles from the Right
- Some commentators argue that America needs to ensure some clear guardrails for AI chatbots, since kids are especially vulnerable to getting addicted to bots that pretend to care about them—and can use the info they’re given to personalize responses, creating a positive feedback loop that keeps users coming back.
- Others contend that state governments should pass laws that strongly rein in the power of large AI companies, since they have a long history of putting growth and engagement metrics ahead of user well-being.
- “Parents’ horror stories are a wake-up call: Protect kids from addictive AI” –NY Post Editorial Board
- “We can’t trust Meta to protect children” –Tiffany Smiley, Washington Examiner
- “AI could rewire the next generation of children” –Dana Suskind, The Hill
Share this!
You've made it this far...
Let's make our relationship official, no 💍 or elaborate proposal required. Learn and stay entertained, for free.👇
All of our news is 100% free and you can unsubscribe anytime; the quiz takes ~10 seconds to complete