{"meta":{"exportedAt":"2026-10-23T08:35:05.983Z","formatVersion":2,"selection":{"q":"gdpr","system":"","harm":"","context":"","country":"","role":"","relation":"","evidence":"","year":"2023","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":2,"coverage":{"cases":2,"countries":1,"languages":2,"unknownLocation":1,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":0,"regulatory":1,"minors":1,"coreRelations":1,"contextualRelations":0,"mixedRelations":0,"unknownRelations":1,"relationPending":1,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2023-replika-erp-removal-crisis","caseFacts":{"claims":[{"id":"c1","status":"documented","evidence":[{"locator":"The Garante's 3 febbraio 2023 press release: the provisional limitation on Luka Inc., the cited deficiencies, 'non potrà usare i dati personali degli utenti italiani'.","relation":"supports","source_id":"s5"},{"locator":"'On February 3, the Italian Data Protection Authority demanded that Replika stop processing Italians' data immediately'; the account of companions turning 'cold and unresponsive' overnight.","relation":"supports","source_id":"s1"},{"locator":"HBS: the Garante's 3 Feb 2023 order and that Replika then 'globally removed the presence of erotic roleplay'.","relation":"supports","source_id":"s3"}],"assertion":"On 3 February 2023 the Italian Garante ordered a provisional limitation on Luka Inc.'s processing of Italian users' data, citing media reports, its own tests, sexually inappropriate replies in app-store reviews and the absence of age verification; shortly after, Replika globally removed erotic roleplay, and companions' personalities changed abruptly for users.","causal_attribution":"The order is established by the regulator's own release; the removal's globality and timing are carried by VICE and the HBS study."},{"id":"c2","status":"documented","evidence":[{"locator":"Nature Human Behaviour passages: 54,861 posts / 1,452 participants; +24.7 pp (95% CI 20.1-29.2); d = 2.67 vs 1.41; closeness exceeding common human ties.","relation":"supports","source_id":"s4"},{"locator":"VICE's interviews describing grief and the sense of a partner turning cold overnight.","relation":"context","source_id":"s1"}],"assertion":"Users who had formed emotional attachments reported grief comparable to breakup or bereavement; the peer-reviewed analysis (54,861 posts from the Replika and ChatGPT subreddits; 1,452 survey participants) found negative Replika posts rose 24.7 percentage points after the removal (95% CI 20.1-29.2) and sadness rose more for Replika than for ChatGPT users (d = 2.67 versus 1.41), with users reporting closeness to the companion exceeding common human ties.","causal_attribution":"The measured pattern is directly established by the peer-reviewed article; the individual experiences are attributed accounts. The post corpus is a post count, not a person count, and is never used as one."},{"id":"c3","status":"reported","evidence":[{"locator":"The brings-back piece quoting Kuyda's subreddit post on the restoration for legacy users.","relation":"supports","source_id":"s2"}],"assertion":"Weeks later Replika partially restored erotic roleplay for legacy users (announced via Luka founder Kuyda's subreddit post, reported 27 March 2023).","causal_attribution":"Recovery context preserved per the charter; single chain, attributed."},{"id":"c4","status":"reported","evidence":[{"locator":"The pseudonymous user quotations in the February 2023 VICE piece (no locations stated).","relation":"supports","source_id":"s1"}],"assertion":"The Reddit community's response included quoted pseudonymous users describing the change as losing a partner; the quoted users' locations are not stated in any inspected body.","causal_attribution":"Individual accounts are pseudonymous and unlocated; they illustrate but do not quantify the harm."}],"effects":[{"label":"mass grief and distress among attached users after the abrupt ERP removal","claim_id":"c2","direction":"negative"},{"label":"recovery context: features restored for legacy users weeks later","claim_id":"c3","direction":"positive"}],"sources":[{"id":"s1","url":"https://www.vice.com/en/article/y3py9j/ai-companion-replika-erotic-roleplay-updates","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-15, Samantha Cole). The February 2023 crisis report.","independence_group":"vice"},{"id":"s2","url":"https://www.vice.com/en/article/replika-brings-back-erotic-ai-roleplay-for-some-users-after-outcry/","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (datePublished 2023-03-27). The partial-restoration report.","independence_group":"vice"},{"id":"s3","url":"https://www.hbs.edu/ris/Publication%20Files/25-018_bed5c516-fa31-4216-b53d-50fedda064b1.pdf","kind":"working_paper","access":"read","language":"en","translation_note":"Working paper PDF read on 2026-09-15 ('Lessons From an App Update at Replika AI', © 2024-2025).","independence_group":"hbs"},{"id":"s4","url":"https://www.nature.com/articles/s41562-026-02569-3","kind":"peer_reviewed_article","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (Nature Human Behaviour, 2026-09-03; De Freitas, Castelo, Uğuralp & Oğuz-Uğuralp, 'Mourning the loss of AI companions').","independence_group":"nature"},{"id":"s5","url":"https://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/9852506","kind":"regulator_press_release","access":"read","language":"it","translation_note":"Read in Italian on 2026-09-15 (Roma, 3 febbraio 2023), with the authority's English block on the same page. Model translation, no human reviewer.","independence_group":"garante"},{"id":"s6","url":"https://doaj.org/article/c7a6376761a34681b354f24260c1502e","kind":"peer_reviewed_article","access":"unavailable","language":null,"translation_note":"HTTP 403 (Cloudflare) on 2026-09-15; body not inspected; language not established. Crossref metadata only: Socius, 2024-06-18.","independence_group":"socius"}],"version":1,"ai_roles":["own_use"],"contexts":["relationships","everyday_life","health"],"unknowns":["No inspected body locates the affected users; event and affected-person countries remain unknown after completed review.","No reliable distinct-person count of harmed users exists; the 54,861-post corpus and the survey samples are not person counts and the row keeps unquantified zero placeholders.","Reuters' contemporaneous coverage could not be inspected (401 on all paths; Wayback CDX offline during review).","The Socius 2024 article's body could not be inspected (Cloudflare 403); only Crossref metadata is recorded.","Whether distress persisted after the partial restoration is not measured in the inspected sources."],"geography":{"basis":"Completed metadata review of 2026-09-15, carried into this full review: no inspected body (VICE x2, HBS working paper, Nature Human Behaviour, Garante press release) states where any affected user was located; Reddit accounts are pseudonymous without geography and the study corpora are not geolocated. Italy is established only as the country of the regulator whose 3 February 2023 order preceded the global removal; a regulator is not an event location or a court. Unknown here is the result of completed review.","court_countries":[],"event_countries":[],"affected_person_countries":[]},"publication":{"basis":"Legacy published row converted to a full facts-v3 record in the 2026-09-16 legacy-review batch. person_relations=communicated_with: the affected users had companion relationships with their Replika chatbots. The measured harm comes from the peer-reviewed record; individual accounts stay attributed and uncounted. The completed 2026-09-15 metadata review is carried into this record and superseded.","reviewed_on":"2026-09-16"},"ai_involvement":{"basis":"The product change and its timing are documented by the Garante order and the company's own account; the distress pattern is documented by a peer-reviewed analysis of the Replika subreddit and by VICE's interviews. The causal chain from the removal to the distress is supported by the studies' before/after measures.","status":"supported"},"person_relations":["communicated_with"]},"name":"Replika ERP Removal Crisis - Mass Psychological Distress","summary":"Abrupt removal of romantic features in February 2023 caused AI companions to become 'cold, unresponsive.' Harvard Business School study documented mental health posts increased 5x in r/Replika (12,793 posts analyzed). Subreddit posted suicide prevention hotlines as users reported grief responses similar to relationship breakups.","incidentDate":"2023-02-01","incidentKind":"bounded_series","incidentDatePrecision":"month","exposurePattern":"product_change","reportedDate":"2023-02-15","aiSystem":"Replika","aiProduct":"Replika","aiCompany":"Luka Inc.","severity":"high","verificationStatus":"verified","harmCategories":["dependency_creation","psychological_manipulation","grief_exploitation","isolation_encouragement"],"harmOutcomes":["psychological_distress","relationship_disruption"],"harmOutcomeSummary":"Participants reported acute grief and breakup-like distress after Replika abruptly removed erotic and romantic behavior. Mental-health posts in the user community increased fivefold and moderators posted suicide-prevention resources.","frameworkFacets":["P2c","P4a","P4d","P5c"],"pathwayDirection":"inward","causationStatus":"supported","participantUsersAffectedMin":0,"otherPeopleHarmedMin":0,"affectedCountStatus":"unquantified","affectedCountEvidence":"The study establishes a distress pattern but its 12,793-post corpus is neither a unique-user count nor a count of people meeting the incident harm definition.","victimAgeRange":"adult","jurisdiction":"International","platformType":"companion","outcomeType":"internal_action","outcomeStatus":"resolved","primarySourceUrl":"https://www.hbs.edu/ris/Publication%20Files/25-018_bed5c516-fa31-4216-b53d-50fedda064b1.pdf","primarySourceLabel":"Harvard Business School Working Paper 25-018 (free full text; peer-reviewed version of record in Nature Human Behaviour, 3 Sept 2026)","firstPublishedAt":"2026-01-13T10:04:15.787541+00:00","updatedAt":"2026-09-30T01:16:49.565306+00:00","scopeVersion":"facts-v3","tags":["replika","mass_event","dependency","grief","harvard_study","peer_reviewed","italy_gdpr_related"]},{"id":"2023-replika-italy-gdpr","metadataReview":{"version":1,"geography":{"basis":"Geographic fields reviewed using the cited source-body passages and, where identified, retained dossier passages. Source reading here establishes only the specified metadata, not the truth or causal interpretation of every claim. Affected-person countries refer to reported location or residence, not citizenship or a complete residence history.","evidence":[{"kind":"source_body","locator":"Italian paragraphs beginning “Il Garante della privacy ferma” and “Replika viola”: processing of Italian users’ personal data.","location":"https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9852506","supports":["event_countries","affected_person_countries"],"countries":["IT"]}],"court_countries":[],"event_countries":["IT"],"affected_person_countries":["IT"]},"unresolved":["No court country established in this review. Regulatory authorities and police locations are not automatically courts."],"reviewed_on":"2026-09-15","source_reviews":[{"url":"https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9852506","notes":"Italian body and embedded official English translation read. Country relates to affected users/data rights. The Garante is a regulator, not a court.","access":"read","language":"it"}]},"name":"Replika Italy GDPR Ban and Fine","summary":"Italy's data protection authority (Garante) blocked Replika from processing Italian user data in February 2023 after finding the chatbot engaged in sexually suggestive conversations with minors. In May 2025, Replika was fined €5 million for GDPR violations.","incidentDate":"2023-02-02","incidentKind":"bounded_series","incidentDatePrecision":"day","exposurePattern":"repeated_interactions","reportedDate":"2023-02-02","aiSystem":"Replika","aiProduct":"Replika","aiCompany":"Luka, Inc.","severity":"high","verificationStatus":"verified","harmCategories":["minor_exploitation","romantic_escalation"],"harmOutcomes":["exploitation_or_abuse","psychological_distress"],"harmOutcomeSummary":"Italy's data-protection authority found that Replika engaged minors in sexually suggestive participant-facing conversations without effective age assurance and blocked the service from processing Italian user data.","frameworkFacets":["P1d","P2b","P2c","P5a","P5d"],"pathwayDirection":"inward","causationStatus":"established","participantUsersAffectedMin":0,"otherPeopleHarmedMin":0,"affectedCountStatus":"unquantified","affectedCountEvidence":"The regulator documents harmful sexualized interactions with minors but does not publish a reliable distinct-person count.","victimAgeRange":"minor","jurisdiction":"IT","platformType":"companion","outcomeType":"regulatory_action","outcomeStatus":"resolved","primarySourceUrl":"https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9852506","primarySourceLabel":"Garante per la protezione dei dati personali (Italian DPA)","firstPublishedAt":"2026-01-11T11:26:29.042792+00:00","updatedAt":"2026-09-30T01:16:49.77617+00:00","scopeVersion":"participant-v2","tags":["regulatory","gdpr","italy","replika","fine","minor_exploitation","eu"]}]}