{"meta":{"exportedAt":"2026-09-30T07:31:21.413Z","formatVersion":2,"selection":{"q":"mental-health","system":"","harm":"","context":"","country":"","role":"","relation":"","evidence":"","year":"2023","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":1,"coverage":{"cases":1,"countries":0,"languages":2,"unknownLocation":1,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":0,"regulatory":0,"minors":0,"coreRelations":1,"contextualRelations":0,"mixedRelations":0,"unknownRelations":0,"relationPending":0,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2023-replika-erp-removal-crisis","caseFacts":{"claims":[{"id":"c1","status":"documented","evidence":[{"locator":"The Garante's 3 febbraio 2023 press release: the provisional limitation on Luka Inc., the cited deficiencies, 'non potrà usare i dati personali degli utenti italiani'.","relation":"supports","source_id":"s5"},{"locator":"'On February 3, the Italian Data Protection Authority demanded that Replika stop processing Italians' data immediately'; the account of companions turning 'cold and unresponsive' overnight.","relation":"supports","source_id":"s1"},{"locator":"HBS: the Garante's 3 Feb 2023 order and that Replika then 'globally removed the presence of erotic roleplay'.","relation":"supports","source_id":"s3"}],"assertion":"On 3 February 2023 the Italian Garante ordered a provisional limitation on Luka Inc.'s processing of Italian users' data, citing media reports, its own tests, sexually inappropriate replies in app-store reviews and the absence of age verification; shortly after, Replika globally removed erotic roleplay, and companions' personalities changed abruptly for users.","causal_attribution":"The order is established by the regulator's own release; the removal's globality and timing are carried by VICE and the HBS study."},{"id":"c2","status":"documented","evidence":[{"locator":"Nature Human Behaviour passages: 54,861 posts / 1,452 participants; +24.7 pp (95% CI 20.1-29.2); d = 2.67 vs 1.41; closeness exceeding common human ties.","relation":"supports","source_id":"s4"},{"locator":"VICE's interviews describing grief and the sense of a partner turning cold overnight.","relation":"context","source_id":"s1"}],"assertion":"Users who had formed emotional attachments reported grief comparable to breakup or bereavement; the peer-reviewed analysis (54,861 posts from the Replika and ChatGPT subreddits; 1,452 survey participants) found negative Replika posts rose 24.7 percentage points after the removal (95% CI 20.1-29.2) and sadness rose more for Replika than for ChatGPT users (d = 2.67 versus 1.41), with users reporting closeness to the companion exceeding common human ties.","causal_attribution":"The measured pattern is directly established by the peer-reviewed article; the individual experiences are attributed accounts. The post corpus is a post count, not a person count, and is never used as one."},{"id":"c3","status":"reported","evidence":[{"locator":"The brings-back piece quoting Kuyda's subreddit post on the restoration for legacy users.","relation":"supports","source_id":"s2"}],"assertion":"Weeks later Replika partially restored erotic roleplay for legacy users (announced via Luka founder Kuyda's subreddit post, reported 27 March 2023).","causal_attribution":"Recovery context preserved per the charter; single chain, attributed."},{"id":"c4","status":"reported","evidence":[{"locator":"The pseudonymous user quotations in the February 2023 VICE piece (no locations stated).","relation":"supports","source_id":"s1"}],"assertion":"The Reddit community's response included quoted pseudonymous users describing the change as losing a partner; the quoted users' locations are not stated in any inspected body.","causal_attribution":"Individual accounts are pseudonymous and unlocated; they illustrate but do not quantify the harm."}],"effects":[{"label":"mass grief and distress among attached users after the abrupt ERP removal","claim_id":"c2","direction":"negative"},{"label":"recovery context: features restored for legacy users weeks later","claim_id":"c3","direction":"positive"}],"sources":[{"id":"s1","url":"https://www.vice.com/en/article/y3py9j/ai-companion-replika-erotic-roleplay-updates","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-15, Samantha Cole). The February 2023 crisis report.","independence_group":"vice"},{"id":"s2","url":"https://www.vice.com/en/article/replika-brings-back-erotic-ai-roleplay-for-some-users-after-outcry/","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (datePublished 2023-03-27). The partial-restoration report.","independence_group":"vice"},{"id":"s3","url":"https://www.hbs.edu/ris/Publication%20Files/25-018_bed5c516-fa31-4216-b53d-50fedda064b1.pdf","kind":"working_paper","access":"read","language":"en","translation_note":"Working paper PDF read on 2026-09-15 ('Lessons From an App Update at Replika AI', © 2024-2025).","independence_group":"hbs"},{"id":"s4","url":"https://www.nature.com/articles/s41562-026-02569-3","kind":"peer_reviewed_article","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (Nature Human Behaviour, 2026-09-03; De Freitas, Castelo, Uğuralp & Oğuz-Uğuralp, 'Mourning the loss of AI companions').","independence_group":"nature"},{"id":"s5","url":"https://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/9852506","kind":"regulator_press_release","access":"read","language":"it","translation_note":"Read in Italian on 2026-09-15 (Roma, 3 febbraio 2023), with the authority's English block on the same page. Model translation, no human reviewer.","independence_group":"garante"},{"id":"s6","url":"https://doaj.org/article/c7a6376761a34681b354f24260c1502e","kind":"peer_reviewed_article","access":"unavailable","language":null,"translation_note":"HTTP 403 (Cloudflare) on 2026-09-15; body not inspected; language not established. Crossref metadata only: Socius, 2024-06-18.","independence_group":"socius"}],"version":1,"ai_roles":["own_use"],"contexts":["relationships","everyday_life","health"],"unknowns":["No inspected body locates the affected users; event and affected-person countries remain unknown after completed review.","No reliable distinct-person count of harmed users exists; the 54,861-post corpus and the survey samples are not person counts and the row keeps unquantified zero placeholders.","Reuters' contemporaneous coverage could not be inspected (401 on all paths; Wayback CDX offline during review).","The Socius 2024 article's body could not be inspected (Cloudflare 403); only Crossref metadata is recorded.","Whether distress persisted after the partial restoration is not measured in the inspected sources."],"geography":{"basis":"Completed metadata review of 2026-09-15, carried into this full review: no inspected body (VICE x2, HBS working paper, Nature Human Behaviour, Garante press release) states where any affected user was located; Reddit accounts are pseudonymous without geography and the study corpora are not geolocated. Italy is established only as the country of the regulator whose 3 February 2023 order preceded the global removal; a regulator is not an event location or a court. Unknown here is the result of completed review.","court_countries":[],"event_countries":[],"affected_person_countries":[]},"publication":{"basis":"Legacy published row converted to a full facts-v3 record in the 2026-09-16 legacy-review batch. person_relations=communicated_with: the affected users had companion relationships with their Replika chatbots. The measured harm comes from the peer-reviewed record; individual accounts stay attributed and uncounted. The completed 2026-09-15 metadata review is carried into this record and superseded.","reviewed_on":"2026-09-16"},"ai_involvement":{"basis":"The product change and its timing are documented by the Garante order and the company's own account; the distress pattern is documented by a peer-reviewed analysis of the Replika subreddit and by VICE's interviews. The causal chain from the removal to the distress is supported by the studies' before/after measures.","status":"supported"},"person_relations":["communicated_with"]},"name":"Replika ERP Removal Crisis - Mass Psychological Distress","summary":"Abrupt removal of romantic features in February 2023 caused AI companions to become 'cold, unresponsive.' Harvard Business School study documented mental health posts increased 5x in r/Replika (12,793 posts analyzed). Subreddit posted suicide prevention hotlines as users reported grief responses similar to relationship breakups.","incidentDate":"2023-02-01","incidentKind":"bounded_series","incidentDatePrecision":"month","exposurePattern":"product_change","reportedDate":"2023-02-15","aiSystem":"Replika","aiProduct":"Replika","aiCompany":"Luka Inc.","severity":"high","verificationStatus":"verified","harmCategories":["dependency_creation","psychological_manipulation","grief_exploitation","isolation_encouragement"],"harmOutcomes":["psychological_distress","relationship_disruption"],"harmOutcomeSummary":"Participants reported acute grief and breakup-like distress after Replika abruptly removed erotic and romantic behavior. Mental-health posts in the user community increased fivefold and moderators posted suicide-prevention resources.","frameworkFacets":["P2c","P4a","P4d","P5c"],"pathwayDirection":"inward","causationStatus":"supported","participantUsersAffectedMin":0,"otherPeopleHarmedMin":0,"affectedCountStatus":"unquantified","affectedCountEvidence":"The study establishes a distress pattern but its 12,793-post corpus is neither a unique-user count nor a count of people meeting the incident harm definition.","victimAgeRange":"adult","jurisdiction":"International","platformType":"companion","outcomeType":"internal_action","outcomeStatus":"resolved","primarySourceUrl":"https://www.hbs.edu/ris/Publication%20Files/25-018_bed5c516-fa31-4216-b53d-50fedda064b1.pdf","primarySourceLabel":"Harvard Business School Working Paper 25-018 (free full text; peer-reviewed version of record in Nature Human Behaviour, 3 Sept 2026)","firstPublishedAt":"2026-01-13T10:04:15.787541+00:00","updatedAt":"2026-09-30T01:16:49.565306+00:00","scopeVersion":"facts-v3","tags":["replika","mass_event","dependency","grief","harvard_study","peer_reviewed","italy_gdpr_related"]}]}