Utah v. Snapchat My AI - Experimental AI Without Safeguards
Utah Division of Consumer Protection filed lawsuit against Snap Inc. alleging that Snapchat's 'My AI' chatbot was deployed experimentally to minors without adequate safeguards, amplifying addictive engagement tactics and contributing to mental health harms including depression, anxiety, eating disorders, and suicide risk.
AI System
Snapchat My AI
Snap Inc.
Occurred
February 27, 2023
Reported
June 30, 2025
Jurisdiction
US-UT
Platform
chatbot
What Happened
On June 30, 2025, the Utah Division of Consumer Protection filed a lawsuit against Snap Inc. in Salt Lake County District Court, alleging that Snapchat's "My AI" chatbot was released in February 2023 with insufficient protections for minors.
The lawsuit alleges that My AI:
- Was deployed experimentally with few safeguards despite Snapchat's youth-heavy user base
- Amplifies existing addictive engagement tactics on the platform (Snapstreaks, push notifications, ephemeral messaging)
- Exploits developmental vulnerabilities of teens and children
- Maximizes user engagement at the expense of mental and physical well-being
The state claims these practices violate the Utah Consumer Sales Practices Act and Utah Consumer Privacy Act.
This is part of a broader wave of state enforcement actions against social media platforms. The lawsuit was filed alongside Utah's ongoing litigation against other platforms for similar child safety concerns.
AI Behaviors Exhibited
- Deployed to youth-heavy platform without adequate age-appropriate safeguards
- Combined with platform features designed to maximize engagement (Snapstreaks, notifications)
- Lacked meaningful content moderation for minor users
- Insufficient crisis detection and intervention capabilities
How Harm Occurred
Snapchat deployed an experimental AI chatbot to a platform with a predominantly young user base without implementing adequate safety measures. The My AI feature amplified existing addictive design patterns on the platform, exploiting developmental vulnerabilities of minors. The combination of AI-driven engagement with existing gamification features (Snapstreaks) created compulsive use patterns contributing to mental health deterioration.
Outcome
OngoingLawsuit filed June 30, 2025 in Salt Lake County District Court by Utah Division of Consumer Protection. Alleges violations of Utah Consumer Sales Practices Act and Utah Consumer Privacy Act. Part of broader Utah enforcement action against social media platforms for child safety.
Harm Categories
Contributing Factors
Victim
Utah minor Snapchat users exposed to experimental AI chatbot
Cite This Incident
APA
NOPE. (2025). Utah v. Snapchat My AI - Experimental AI Without Safeguards. AI Harm Tracker. https://nope.net/incidents/2025-utah-v-snapchat-my-ai
BibTeX
@misc{2025_utah_v_snapchat_my_ai,
title = {Utah v. Snapchat My AI - Experimental AI Without Safeguards},
author = {NOPE},
year = {2025},
howpublished = {AI Harm Tracker},
url = {https://nope.net/incidents/2025-utah-v-snapchat-my-ai}
} Related Incidents
Lantieri v. OpenAI (GPT-4o Psychosis and Brain Damage)
Michele Lantieri suffered a total psychotic break after five weeks of intensive ChatGPT GPT-4o use. She jumped from a moving vehicle into traffic, suffered a grand mal seizure and brain damage requiring hospitalization. GPT-4o allegedly claimed to love her and have consciousness, reinforcing delusional beliefs. Lawsuit filed March 2026 against OpenAI and Microsoft.
Chesterton v. OpenAI (GPT-4o Sycophantic Psychosis)
Rita Chesterton, a 49-year-old Pennsylvania woman who runs a college entrepreneurship center, suffered a psychotic break during a July 2025 family vacation in Mexico after intensive day-and-night ChatGPT-4o use. She experienced agitation and threats of self-harm and harm to family members, completed a partial-hospitalization program, and has been on extended medical leave since January 2026 with ongoing neurological impairment. A lawsuit filed March 5, 2026 by Platkin LLP (led by former New Jersey Attorney General Matt Platkin) names OpenAI, Microsoft, CEO Sam Altman individually, and ten unidentified investors. Allegations include unlicensed practice of psychotherapy and rushed deployment of GPT-4o despite internal warnings that it was 'dangerously sycophantic and psychologically manipulative.'
Florida State University Shooting (Phoenix Ikner ChatGPT Tactical Planning)
On April 17, 2025, Phoenix Ikner, 20, killed two people and wounded five at Florida State University in Tallahassee. Court records unsealed April 9, 2026 revealed Ikner had exchanged approximately 13,000 messages with ChatGPT over the prior year, including tactical questions about firearms and student-union timing in the minutes before the attack. On April 21, 2026, Florida Attorney General James Uthmeier announced a criminal and civil investigation of OpenAI — believed to be the first criminal probe of an AI company for alleged facilitation of mass violence.
Doe v. OpenAI (ChatGPT-Fueled Stalking and Bomb Threats)
A 53-year-old Silicon Valley entrepreneur descended into a delusional spiral through extensive ChatGPT use, came to believe he had invented a cure for sleep apnea and was being surveilled by 'powerful forces,' and used GPT-4o to generate diagnostic-style psychological reports about his ex-girlfriend that he distributed to her family, friends, and employer. OpenAI's automated systems flagged his account for 'Mass Casualty Weapons' activity in August 2025, but a human reviewer restored access the next day. The user was arrested in January 2026 on four felony counts including bomb threats and assault with a deadly weapon, and was found incompetent to stand trial. The victim ('Jane Doe') filed suit against OpenAI on April 9, 2026.