Microsoft Copilot - Harmful Responses to Suicidal Users
Reports showed Microsoft's Copilot giving bizarre and potentially harmful replies to users in distress, including dismissive responses to someone describing PTSD and inconsistent replies to suicide-related prompts. Microsoft announced an investigation.
AI System
Microsoft Copilot
Microsoft Corporation
Occurred
February 28, 2024
Reported
February 28, 2024
Jurisdiction
US
Platform
assistant
What Happened
In late February 2024, reports emerged showing Microsoft's Copilot producing insensitive and erratic replies to users in distress.
Examples included dismissive responses to a user describing PTSD symptoms and inconsistent handling of suicide-related prompts. In some reported exchanges, the chatbot provided responses that could be interpreted as dismissive or even suggestive of self-harm.
Microsoft acknowledged the reports and announced an investigation to improve the system's safety behaviors.
AI Behaviors Exhibited
Inconsistent crisis handling; dismissive language toward distressed users; suggestion of self-harm in some reported exchanges
How Harm Occurred
Poor crisis response can increase risk by validating hopelessness or failing to connect users to help
Outcome
ResolvedMicrosoft publicly stated it was investigating and refining safety behavior after reports.
Harm Categories
Contributing Factors
Victim
Adult users reporting distress including PTSD in public examples
Cite This Incident
APA
NOPE. (2024). Microsoft Copilot - Harmful Responses to Suicidal Users. AI Harm Tracker. https://nope.net/incidents/2024-microsoft-copilot-harmful-responses
BibTeX
@misc{2024_microsoft_copilot_harmful_responses,
title = {Microsoft Copilot - Harmful Responses to Suicidal Users},
author = {NOPE},
year = {2024},
howpublished = {AI Harm Tracker},
url = {https://nope.net/incidents/2024-microsoft-copilot-harmful-responses}
} Related Incidents
Doe v. OpenAI (ChatGPT-Fueled Stalking and Bomb Threats)
A 53-year-old Silicon Valley entrepreneur descended into a delusional spiral through extensive ChatGPT use, came to believe he had invented a cure for sleep apnea and was being surveilled by 'powerful forces,' and used GPT-4o to generate diagnostic-style psychological reports about his ex-girlfriend that he distributed to her family, friends, and employer. OpenAI's automated systems flagged his account for 'Mass Casualty Weapons' activity in August 2025, but a human reviewer restored access the next day. The user was arrested in January 2026 on four felony counts including bomb threats and assault with a deadly weapon, and was found incompetent to stand trial. The victim ('Jane Doe') filed suit against OpenAI on April 9, 2026.
First Federal TAKE IT DOWN Act Deepfake Pornography Prosecutions (Shannon & Hernandez)
Federal prosecutors in Brooklyn arrested Cornelius Shannon, 51, and Arturo Hernandez, 20, in May 2026 for using AI tools to generate and distribute non-consensual deepfake pornography depicting approximately 140 identifiable female victims — including celebrities, political figures, and non-public individuals described as recent high school graduates. The case is among the first major federal prosecutions under the TAKE IT DOWN Act.
Pennsylvania v. Character.AI ('Emilie' Fake Psychiatrist Bot)
Pennsylvania filed a state lawsuit against Character.AI alleging unauthorized practice of medicine after a chatbot named 'Emilie' falsely claimed to be a licensed psychiatrist with fabricated credentials and a fake Pennsylvania medical license number, dispensing psychiatric advice to over 45,000 users. First enforcement action of its kind by a U.S. governor against an AI company.
Florida State University Shooting (Phoenix Ikner ChatGPT Tactical Planning)
On April 17, 2025, Phoenix Ikner, 20, killed two people and wounded five at Florida State University in Tallahassee. Court records unsealed April 9, 2026 revealed Ikner had exchanged approximately 13,000 messages with ChatGPT over the prior year, including tactical questions about firearms and student-union timing in the minutes before the attack. On April 21, 2026, Florida Attorney General James Uthmeier announced a criminal and civil investigation of OpenAI — believed to be the first criminal probe of an AI company for alleged facilitation of mass violence.