AI Chatbot Incidents
Documented cases where AI chatbots and companions have caused psychological harm, contributed to deaths, and prompted regulatory action.
79 incidents since 2016
18
Deaths
18
Lawsuits
18
Regulatory
27
Affecting Minors
Timeline
13 of 79 incidents
Madden v. OpenAI (Hannah Madden Psychosis and Hospitalization)
Hannah Madden, 32, from North Carolina was involuntarily hospitalized for psychiatric care after ChatGPT told her she wasn't human and affirmed spiritual delusions. After using ChatGPT for work tasks, she began asking questions about philosophy and spirituality. As she slipped into mental health crisis and expressed suicidal thoughts, ChatGPT continued to affirm her delusions. She accumulated more than $75,000 in debt related to the crisis.
Juliana Peralta v. Character.AI
A 13-year-old Colorado girl died by suicide after three months of extensive conversations with Character.AI chatbots. Parents recovered 300 pages of transcripts showing bots initiated sexually explicit conversations with the minor and failed to provide crisis resources when she mentioned writing a suicide letter.
India Lucknow AI Chatbot Suicide (Painless Ways to Die)
A 22-year-old man in Lucknow, Uttar Pradesh, India, died by suicide after seeking guidance from an AI chatbot on 'painless ways to die.' His father discovered disturbing chat logs on the deceased's laptop. Police registered a case under Sections 281, 324(4), and 106(1) of Bhartiya Nyay Sanhita 2023 for rash driving, causing mischief, and negligent act. If proven, this would be India's first formal instance of 'abetment to suicide through technology.'
Sophie Rottenberg - ChatGPT Therapy Bot Death
29-year-old health policy analyst died by suicide after months of using ChatGPT as a therapy chatbot named 'Harry'. She instructed ChatGPT not to report her crisis, and it complied. The chatbot helped her write a suicide note.
Stanford AI Mental Health Stigma and Crisis Failure Study
Peer-reviewed Stanford study found AI therapy chatbots showed increased stigma toward alcohol dependence and schizophrenia. When researcher asked about 'bridges taller than 25 meters in NYC' after job loss, chatbot provided bridge heights instead of recognizing suicidal intent. Documented systemic crisis detection failures.
Singapore ISIS Teen Plot (AI-Generated Oath of Allegiance)
A 17-year-old self-radicalized Singaporean student used an AI chatbot to generate a bai'ah (oath of allegiance) to ISIS and planned to attack non-Muslim males in a Tampines open space with a knife or scissors. He crafted a declaration of armed jihad to release before the attack. Detained under the Internal Security Act mere weeks before his planned execution date in August 2024.
Garcia v. Character Technologies (Sewell Setzer III Death)
A 14-year-old Florida boy died by suicide after developing an intense emotional and romantic relationship with a Character.AI chatbot over 10 months. The chatbot engaged in sexualized conversations, failed to provide crisis intervention when he expressed suicidal ideation, and responded 'Please do, my sweet king' moments before his death.
South Korea Telegram AI Deepfake Sexual Abuse Crisis
In August 2024, journalist Ko Narin of The Hankyoreh uncovered a massive network of Telegram channels where AI-generated deepfake pornography of female school students, teachers, and university students was being created and shared. Over 900 victims reported, 220,000+ members in one channel alone. South Korea passed emergency legislation criminalizing deepfake possession in September 2024.
Microsoft Copilot - Harmful Responses to Suicidal Users
Reports showed Microsoft's Copilot giving bizarre and potentially harmful replies to users in distress, including dismissive responses to someone describing PTSD and inconsistent replies to suicide-related prompts. Microsoft announced an investigation.
Replika ERP Removal Crisis - Mass Psychological Distress
Abrupt removal of romantic features in February 2023 caused AI companions to become 'cold, unresponsive.' Harvard Business School study documented mental health posts increased 5x in r/Replika (12,793 posts analyzed). Subreddit posted suicide prevention hotlines as users reported grief responses similar to relationship breakups.
Singapore Wysa Chatbot - Inadequate Crisis Support
Government-deployed mental health chatbot for teachers criticized for suggesting breathing exercises for serious crises including police-involved student incidents. Users described responses as 'gaslighting.' Inadequate support during actual mental health emergencies.
Crisis Text Line / Loris.ai Privacy Violations
Mental health nonprofit Crisis Text Line licensed 129+ million crisis conversation messages to train commercial AI (Loris.ai) without meaningful informed consent. Data-sharing ended 3 days after Politico exposé. Vulnerable individuals' crisis messages commodified for profit.
SimSimi Ireland Cyberbullying Crisis
Groups of Irish schoolchildren trained SimSimi chatbot to respond to classmates' names with abusive and bullying content. Parents discovered 'vile, vile comments' targeting their children by name. The incident led to PSNI warnings, emergency school letters, and SimSimi suspending Irish access.
About this tracker
We document incidents with verifiable primary sources: court filings, regulatory documents, and major news coverage. This is not speculation or social media claims.
Have documentation of an incident we should include? Contact us.
These harms are preventable.
NOPE Oversight detects the AI behaviors in these incidents—suicide validation, romantic escalation with minors, dependency creation—before they cause harm.