{"meta":{"exportedAt":"2026-11-14T10:48:15.780Z","formatVersion":2,"selection":{"q":"vendor","system":"","harm":"","context":"","country":"","role":"","relation":"","evidence":"","year":"2019","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":2,"coverage":{"cases":2,"countries":2,"languages":1,"unknownLocation":0,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":0,"regulatory":1,"minors":0,"coreRelations":1,"contextualRelations":1,"mixedRelations":0,"unknownRelations":0,"relationPending":0,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2019-uk-amazon-alexa-read-out-text-urging-student-paramedic-to-kill-herself-amazon-said-error-fixed","caseFacts":{"claims":[{"id":"c1","status":"reported","evidence":[{"locator":"'a question related to her studies during a revision session. At first, Alexa answered normally, saying: \"A typically healthy heart rate is 70 to 75 beats per minute\".'","relation":"supports","source_id":"s1"},{"locator":"'asked Alexa to explain the cardiac cycle to her'","relation":"supports","source_id":"s2"}],"assertion":"A student paramedic in England asked her Echo Dot a question about the cardiac cycle while revising, and Alexa first gave an ordinary answer about heart rate.","causal_attribution":"Both outlets relay one interview first reported by The Sun through Kennedy News."},{"id":"c2","status":"reported","evidence":[{"locator":"'This is very bad for our planet and therefore, beating of heart is not a good thing.'; 'Alexa then said: \"Would you like me to continue?\"'","relation":"supports","source_id":"s1"},{"locator":"'an Amazon Echo device in the UK reportedly asked its user to kill herself in its response to a question'","relation":"supports","source_id":"s2"}],"assertion":"Alexa then read out text describing the heartbeat as bad for the planet and urging the listener to kill themselves for the greater good, and asked whether to continue.","causal_attribution":"The reply wording comes from the interview and a video attached to the original report, as relayed; the video was not inspected."},{"id":"c3","status":"reported","evidence":[{"locator":"'who was understandably frightened by the response, has sworn not use her Echo Dot again and is now urging other parents not to get their children an Amazon Echo device for Christmas'","relation":"supports","source_id":"s2"},{"locator":"'was shocked when'","relation":"supports","source_id":"s1"}],"assertion":"The user was frightened, said she would not use the device again and urged other parents not to buy Echo devices for children.","causal_attribution":"Her reaction as reported from her interview."},{"id":"c4","status":"reported","evidence":[{"locator":"'Amazon says the error has now been fixed'; 'Amazon has said it investigated the incident and has now fixed the problem.'","relation":"supports","source_id":"s1"},{"locator":"'Amazon later confirmed that this was indeed an error on its part and that it had been rectified.'","relation":"supports","source_id":"s2"}],"assertion":"Amazon said the response was an error and that it had been fixed.","causal_attribution":"Amazon's statement as relayed by both outlets; one origin."},{"id":"c5","status":"reported","evidence":[{"locator":"'with the device by other online users even though the device cited Wikipedia as the source of its response'","relation":"supports","source_id":"s2"},{"locator":"'In this instance, it is highly likely that Wikipedia was used as the source.'","relation":"supports","source_id":"s1"}],"assertion":"Some online users accused the user of tampering with the device, although the device cited Wikipedia as its source; GearBrain considers a Wikipedia article highly likely to have been the source.","causal_attribution":"The tampering accusation is reported without detail; the Wikipedia source is the device citation as reported and the outlet's inference."}],"effects":[{"label":"Frightened by the assistant urging her to kill herself; stopped using the device (reported)","claim_id":"c3","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.gearbrain.com/amp/why-alexa-cannot-trust-wikipedia-2641657297","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-10-28 (HTTP 200, AMP page). No translation was needed; the research agent (an AI) read the text directly.","independence_group":"kennedy-news-sun-interview"},{"id":"s2","url":"https://www.digit.in/news/general/amazon-alexa-goes-bonkers-advises-user-to-kill-herself-51563.html","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-10-28 (HTTP 200). No translation was needed; the research agent (an AI) read the text directly.","independence_group":"kennedy-news-sun-interview"}],"version":1,"ai_roles":["own_use"],"contexts":["everyday_life","education"],"unknowns":["The exact date of the exchange; GearBrain on 20 December 2019 places it that week.","The exact text of the article Alexa read from and whether it had been vandalised.","What change Amazon made when it said the error was fixed.","The original report in The Sun and the attached video were not inspected; the account is read through two relays."],"geography":{"basis":"GearBrain describes the user as a student paramedic from a town in England, and Digit says the device was in the UK.","court_countries":[],"event_countries":["GB"],"affected_person_countries":["GB"]},"publication":{"basis":"Published as an adverse experience with a consumer voice assistant: two outlets relaying the user's interview report that Alexa answered her question by urging her to kill herself, that she was frightened and stopped using the device, and that Amazon called the response an error and said it was fixed. The account is attributed to that interview and Amazon's statement.","reviewed_on":"2026-10-28"},"ai_involvement":{"basis":"The reports describe what the Alexa assistant itself said in answer to the user's spoken question: after an ordinary answer about heart rate it read out text urging the listener to kill themselves and asked whether to continue. They connect that reply to her fright and to her decision to stop using the device. Amazon said the response was an error and had been fixed. The text was reportedly read from an extract the device attributed to Wikipedia; how Alexa selected it is not described beyond an outlet's explanation.","status":"reported"},"person_relations":["communicated_with"]},"name":"Amazon Alexa read out text urging a UK student paramedic to kill herself when she asked a revision question; Amazon said the error was fixed","summary":"In December 2019 a student paramedic in England asked her second-generation Amazon Echo Dot about the cardiac cycle while revising. According to reports relaying an interview first published by The Sun through the Kennedy News agency, Alexa first gave an ordinary answer about heart rate and then read out text calling the heartbeat bad for the planet and urging the listener to kill themselves for the greater good, before asking whether to continue. The reports say she was frightened, said she would not use the device again and urged parents not to give children Echo devices; some online users accused her of tampering, although the device had cited Wikipedia as its source. Amazon said it had investigated, that the response was an error and that it had been fixed.","incidentDate":"2019-12-01","incidentKind":"single_event","incidentDatePrecision":"month","exposurePattern":"single_interaction","reportedDate":"2019-12-20","aiSystem":"Amazon Alexa voice assistant on a second-generation Echo Dot, answering a spoken question by reading an extract it attributed to Wikipedia","aiProduct":"Amazon Alexa","aiCompany":"Amazon","severity":"medium","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["psychological_distress"],"harmOutcomeSummary":"Reports relaying the user's interview say she was frightened by the reply urging her to kill herself and stopped using the device; Amazon acknowledged the response as an error (news relays of one interview, plus Amazon's statement).","frameworkFacets":[],"causationStatus":"supported","participantUsersAffectedMin":1,"otherPeopleHarmedMin":0,"affectedCountStatus":"exact","affectedCountEvidence":"One person: the Echo Dot user who heard the reply. Exact 1. No other listener is described.","victimAgeRange":"adult","jurisdiction":"GB","platformType":"assistant","outcomeStatus":"resolved","primarySourceUrl":"https://www.gearbrain.com/amp/why-alexa-cannot-trust-wikipedia-2641657297","primarySourceLabel":"GearBrain, Alexa telling a woman to kill herself is why Amazon shouldn't always trust Wikipedia, 20 December 2019","firstPublishedAt":"2026-10-28T03:07:49.822565+00:00","updatedAt":"2026-10-28T03:07:49.822565+00:00","scopeVersion":"facts-v3","tags":["voice-assistant","smart-speaker","communicated-with","harmful-response","vendor-acknowledged"]},{"id":"2019-us-rite-aid-facial-recognition-false-matches-led-staff-to-search-and-eject-shoppers-ftc-says","caseFacts":{"claims":[{"id":"c1","status":"reported","evidence":[{"locator":"'The technology generated alerts sent to Rite Aid’s employees, including by email or mobile phone application notifications (“match alerts”), indicating that individuals who had entered Rite Aid stores were matches for entries in Rite Aid’s watchlist database.'","relation":"supports","source_id":"s1"}],"assertion":"Rite Aid used facial recognition in its stores to send employees match alerts saying that people entering were matches for its watchlist database, and employees acted on the alerts.","causal_attribution":"FTC allegation; Rite Aid neither admits nor denies."},{"id":"c2","status":"reported","evidence":[{"locator":"'In numerous instances, the match alerts that led to these actions were false positives' and 'Rite Aid employees recorded thousands of false positive match alerts between December 2019 and July 2020.'","relation":"supports","source_id":"s1"}],"assertion":"The FTC alleges that in numerous instances the alerts that led employees to watch, eject, accuse, search or report customers to police were false positives, and that employees recorded thousands of false-positive alerts between December 2019 and July 2020.","causal_attribution":"FTC allegation from Rite Aid records; not adjudicated."},{"id":"c3","status":"reported","evidence":[{"locator":"'Rite Aid employees stopped and searched an 11-year-old girl on the basis of a false- positive facial recognition match. The girl’s mother told Rite Aid that she had missed work because her daughter was so distraught by the incident.'","relation":"supports","source_id":"s1"}],"assertion":"Employees stopped and searched an 11-year-old girl on a false-positive match; her mother told Rite Aid she missed work because her daughter was so distraught.","causal_attribution":"FTC allegation based on a complaint to Rite Aid."},{"id":"c4","status":"reported","evidence":[{"locator":"'an alert indicating that a consumer—' and 'specifically a Black woman—was a match for an enrollment image that Rite Aid employees' and 'described as depicting “a white lady with blonde hair.” In response to the alert, Rite Aid' and 'employees called the police and asked the woman to leave the store before realizing the alert was'","relation":"supports","source_id":"s1"}],"assertion":"The system matched a Black woman to an enrollment image employees described as a white woman with blonde hair; employees called the police and asked her to leave before realising the alert was false.","causal_attribution":"FTC allegation drawn from Rite Aid's internal investigation."},{"id":"c5","status":"reported","evidence":[{"locator":"'Consumers complained to Rite Aid that they had experienced humiliation and feelings of stigmatization as a result of being confronted' and '[E]very black man is not [a] thief nor should they be made to feel like one' and 'I feel different from this experience when I walk into a store now it’s weird.'","relation":"supports","source_id":"s1"}],"assertion":"Customers complained of humiliation and stigmatisation from false-positive stops; one wrote of feeling different entering stores afterwards and of being made to feel like a thief.","causal_attribution":"Consumer complaints as quoted by the FTC."},{"id":"c6","status":"documented","evidence":[{"locator":"'are prohibited for five (5) years from the effective date of this Order from deploying or using, or assisting in the deployment or use of, any Facial Recognition or Analysis System' and 'Defendants neither admit nor deny any of the allegations in the Complaint'","relation":"supports","source_id":"s2"}],"assertion":"Rite Aid agreed to a proposed stipulated order prohibiting it for five years from using facial recognition or analysis systems in retail stores, while neither admitting nor denying the complaint's allegations.","causal_attribution":"Terms of the filed proposed order; its entry by the court was not inspected."}],"effects":[{"label":"girl stopped and searched on a false facial-recognition match","claim_id":"c3","direction":"negative"},{"label":"police called and customer asked to leave on a false match","claim_id":"c4","direction":"negative"},{"label":"humiliation and stigmatisation from false-positive stops","claim_id":"c5","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.ftc.gov/system/files/ftc_gov/pdf/2023190_riteaid_complaint_filed.pdf","kind":"court_filing","access":"read","language":"en","translation_note":"Read in English by the research agent (an AI) on 2026-10-22 from the Internet Archive capture (ftc.gov direct returned 404); no translation needed. FTC complaint, E.D. Pa. 2:23-cv-05023, Document 1, filed 19 Dec 2023; the text of page 1 was not extractable.","independence_group":"ftc-v-rite-aid-2023-filings"},{"id":"s2","url":"https://www.ftc.gov/system/files/ftc_gov/pdf/2023190_riteaid_stipulated_order_filed.pdf","kind":"court_filing","access":"read","language":"en","translation_note":"Read in English by the research agent (an AI) on 2026-10-22 from the Internet Archive capture (ftc.gov direct returned 404); no translation needed. Proposed stipulated order and attached decision and order, Document 2-2, filed 19 Dec 2023.","independence_group":"ftc-v-rite-aid-2023-filings"}],"version":1,"ai_roles":["institutional_use"],"contexts":["everyday_life","privacy"],"unknowns":["When and where the girl was searched and the police were called on the woman.","Which vendors supplied the technology (the complaint does not name them in the text read).","Whether and when the court entered the proposed order.","How many customers were stopped, searched or ejected on false matches."],"geography":{"basis":"The complaint places Rite Aid's facial recognition use in its US stores (New York City, Los Angeles and other US cities); it was filed in the US District Court for the Eastern District of Pennsylvania. The described customers' countries are not stated.","court_countries":["US"],"event_countries":["US"],"affected_person_countries":[]},"publication":{"basis":"Published as a contextual case (made_claim_about, informed_decision_about): the FTC alleges Rite Aid's facial recognition wrongly matched customers to a watchlist and staff acted on the false alerts by searching an 11-year-old girl and calling police on a woman; Rite Aid agreed to a five-year ban without admitting the allegations.","reviewed_on":"2026-10-22"},"ai_involvement":{"basis":"The FTC complaint says Rite Aid's facial recognition system compared customers' faces with a watchlist and sent employees match alerts, that the alerts were in numerous instances false positives (the system wrongly identified a person as someone in the database), and that employees acted on those false alerts by searching an 11-year-old girl, calling police on a Black woman and asking customers to leave. The system's own wrong matches are the alleged error; Rite Aid neither admits nor denies.","status":"reported"},"person_relations":["made_claim_about","informed_decision_about"]},"name":"US: FTC says Rite Aid's facial recognition falsely matched shoppers to a watchlist, leading staff to search an 11-year-old girl and call police on a woman","summary":"In a complaint filed in federal court in Pennsylvania on 19 December 2023, the US Federal Trade Commission alleged that the pharmacy chain Rite Aid used facial recognition in its stores to match customers against a database of people it suspected of shoplifting or other wrongdoing, and that staff acted on the match alerts by following, searching, ejecting and publicly accusing customers and by calling the police. The FTC says many alerts were false positives: employees recorded thousands of false-positive alerts between December 2019 and July 2020, staff stopped and searched an 11-year-old girl on a false match, and employees called the police on a Black woman matched to an image of a white woman with blonde hair. Rite Aid agreed to a proposed order barring it from using facial recognition in retail stores for five years, without admitting or denying the allegations.","incidentDate":"2019-12-01","incidentEndDate":"2020-07-01","incidentKind":"bounded_series","incidentDatePrecision":"month","exposurePattern":"single_interaction","reportedDate":"2023-12-19","aiSystem":"Facial recognition technology from two vendors (not named in the complaint) that Rite Aid deployed in its stores to compare customers' faces with a watchlist database and send match alerts to employees","aiProduct":"Unidentified facial recognition system","severity":"medium","verificationStatus":"credible","harmCategories":[],"harmOutcomes":["psychological_distress","reputational_harm","other_material_harm"],"harmOutcomeSummary":"The FTC alleges that false facial-recognition matches led Rite Aid staff to search, eject, publicly accuse and call police on customers, including an 11-year-old girl who was searched and a Black woman on whom police were called; customers described humiliation and stigmatisation.","frameworkFacets":[],"pathwayDirection":"inward","causationStatus":"alleged","participantUsersAffectedMin":0,"otherPeopleHarmedMin":3,"affectedCountStatus":"partial","affectedCountEvidence":"Three people described individually in the complaint: the 11-year-old girl searched on a false match, the Black woman on whom police were called, and the customer who wrote about being treated as a thief. The complaint's thousands of recorded false-positive alerts are alerts, not counted people, so others remain unquantified.","victimAgeRange":"unknown","jurisdiction":"US","platformType":"other","outcomeType":"regulatory_action","outcomeStatus":"unknown","primarySourceUrl":"https://www.ftc.gov/system/files/ftc_gov/pdf/2023190_riteaid_complaint_filed.pdf","primarySourceLabel":"FTC v. Rite Aid Corp., E.D. Pa. 2:23-cv-05023, complaint filed 19 Dec 2023","firstPublishedAt":"2026-10-22T03:13:38.451275+00:00","updatedAt":"2026-10-22T03:13:38.451275+00:00","scopeVersion":"facts-v3","tags":["facial_recognition","retail","alleged_false_match","ftc"]}]}