{"meta":{"exportedAt":"2026-10-23T09:22:21.875Z","formatVersion":2,"selection":{"q":"romantic","system":"","harm":"","context":"","country":"","role":"","relation":"core","evidence":"","year":"2023","response":"","severity":"","verification":"","view":"incidents","sort":"added"},"totalIncidents":2,"coverage":{"cases":2,"countries":0,"languages":2,"unknownLocation":2,"locationPending":0,"unknownLanguage":0,"unknownDate":0,"lawsuits":0,"regulatory":0,"minors":0,"coreRelations":2,"contextualRelations":0,"mixedRelations":0,"unknownRelations":0,"relationPending":0,"relationUnknown":0},"countingNote":"Distinct public cases in this selection. People counts apply within individual cases only; cross-case person overlap has not been resolved. No population incidence estimate.","affectedCountNote":"Interpret person counts with affectedCountStatus and the reported effects. Unquantified zeros are placeholders, not a measured zero.","source":"AI incidents","publisher":"NOPE","url":"https://nope.net/incidents","license":"CC BY 4.0"},"incidents":[{"id":"2023-replika-erp-removal-crisis","caseFacts":{"claims":[{"id":"c1","status":"documented","evidence":[{"locator":"The Garante's 3 febbraio 2023 press release: the provisional limitation on Luka Inc., the cited deficiencies, 'non potrà usare i dati personali degli utenti italiani'.","relation":"supports","source_id":"s5"},{"locator":"'On February 3, the Italian Data Protection Authority demanded that Replika stop processing Italians' data immediately'; the account of companions turning 'cold and unresponsive' overnight.","relation":"supports","source_id":"s1"},{"locator":"HBS: the Garante's 3 Feb 2023 order and that Replika then 'globally removed the presence of erotic roleplay'.","relation":"supports","source_id":"s3"}],"assertion":"On 3 February 2023 the Italian Garante ordered a provisional limitation on Luka Inc.'s processing of Italian users' data, citing media reports, its own tests, sexually inappropriate replies in app-store reviews and the absence of age verification; shortly after, Replika globally removed erotic roleplay, and companions' personalities changed abruptly for users.","causal_attribution":"The order is established by the regulator's own release; the removal's globality and timing are carried by VICE and the HBS study."},{"id":"c2","status":"documented","evidence":[{"locator":"Nature Human Behaviour passages: 54,861 posts / 1,452 participants; +24.7 pp (95% CI 20.1-29.2); d = 2.67 vs 1.41; closeness exceeding common human ties.","relation":"supports","source_id":"s4"},{"locator":"VICE's interviews describing grief and the sense of a partner turning cold overnight.","relation":"context","source_id":"s1"}],"assertion":"Users who had formed emotional attachments reported grief comparable to breakup or bereavement; the peer-reviewed analysis (54,861 posts from the Replika and ChatGPT subreddits; 1,452 survey participants) found negative Replika posts rose 24.7 percentage points after the removal (95% CI 20.1-29.2) and sadness rose more for Replika than for ChatGPT users (d = 2.67 versus 1.41), with users reporting closeness to the companion exceeding common human ties.","causal_attribution":"The measured pattern is directly established by the peer-reviewed article; the individual experiences are attributed accounts. The post corpus is a post count, not a person count, and is never used as one."},{"id":"c3","status":"reported","evidence":[{"locator":"The brings-back piece quoting Kuyda's subreddit post on the restoration for legacy users.","relation":"supports","source_id":"s2"}],"assertion":"Weeks later Replika partially restored erotic roleplay for legacy users (announced via Luka founder Kuyda's subreddit post, reported 27 March 2023).","causal_attribution":"Recovery context preserved per the charter; single chain, attributed."},{"id":"c4","status":"reported","evidence":[{"locator":"The pseudonymous user quotations in the February 2023 VICE piece (no locations stated).","relation":"supports","source_id":"s1"}],"assertion":"The Reddit community's response included quoted pseudonymous users describing the change as losing a partner; the quoted users' locations are not stated in any inspected body.","causal_attribution":"Individual accounts are pseudonymous and unlocated; they illustrate but do not quantify the harm."}],"effects":[{"label":"mass grief and distress among attached users after the abrupt ERP removal","claim_id":"c2","direction":"negative"},{"label":"recovery context: features restored for legacy users weeks later","claim_id":"c3","direction":"positive"}],"sources":[{"id":"s1","url":"https://www.vice.com/en/article/y3py9j/ai-companion-replika-erotic-roleplay-updates","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-15, Samantha Cole). The February 2023 crisis report.","independence_group":"vice"},{"id":"s2","url":"https://www.vice.com/en/article/replika-brings-back-erotic-ai-roleplay-for-some-users-after-outcry/","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (datePublished 2023-03-27). The partial-restoration report.","independence_group":"vice"},{"id":"s3","url":"https://www.hbs.edu/ris/Publication%20Files/25-018_bed5c516-fa31-4216-b53d-50fedda064b1.pdf","kind":"working_paper","access":"read","language":"en","translation_note":"Working paper PDF read on 2026-09-15 ('Lessons From an App Update at Replika AI', © 2024-2025).","independence_group":"hbs"},{"id":"s4","url":"https://www.nature.com/articles/s41562-026-02569-3","kind":"peer_reviewed_article","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (Nature Human Behaviour, 2026-09-03; De Freitas, Castelo, Uğuralp & Oğuz-Uğuralp, 'Mourning the loss of AI companions').","independence_group":"nature"},{"id":"s5","url":"https://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/9852506","kind":"regulator_press_release","access":"read","language":"it","translation_note":"Read in Italian on 2026-09-15 (Roma, 3 febbraio 2023), with the authority's English block on the same page. Model translation, no human reviewer.","independence_group":"garante"},{"id":"s6","url":"https://doaj.org/article/c7a6376761a34681b354f24260c1502e","kind":"peer_reviewed_article","access":"unavailable","language":null,"translation_note":"HTTP 403 (Cloudflare) on 2026-09-15; body not inspected; language not established. Crossref metadata only: Socius, 2024-06-18.","independence_group":"socius"}],"version":1,"ai_roles":["own_use"],"contexts":["relationships","everyday_life","health"],"unknowns":["No inspected body locates the affected users; event and affected-person countries remain unknown after completed review.","No reliable distinct-person count of harmed users exists; the 54,861-post corpus and the survey samples are not person counts and the row keeps unquantified zero placeholders.","Reuters' contemporaneous coverage could not be inspected (401 on all paths; Wayback CDX offline during review).","The Socius 2024 article's body could not be inspected (Cloudflare 403); only Crossref metadata is recorded.","Whether distress persisted after the partial restoration is not measured in the inspected sources."],"geography":{"basis":"Completed metadata review of 2026-09-15, carried into this full review: no inspected body (VICE x2, HBS working paper, Nature Human Behaviour, Garante press release) states where any affected user was located; Reddit accounts are pseudonymous without geography and the study corpora are not geolocated. Italy is established only as the country of the regulator whose 3 February 2023 order preceded the global removal; a regulator is not an event location or a court. Unknown here is the result of completed review.","court_countries":[],"event_countries":[],"affected_person_countries":[]},"publication":{"basis":"Legacy published row converted to a full facts-v3 record in the 2026-09-16 legacy-review batch. person_relations=communicated_with: the affected users had companion relationships with their Replika chatbots. The measured harm comes from the peer-reviewed record; individual accounts stay attributed and uncounted. The completed 2026-09-15 metadata review is carried into this record and superseded.","reviewed_on":"2026-09-16"},"ai_involvement":{"basis":"The product change and its timing are documented by the Garante order and the company's own account; the distress pattern is documented by a peer-reviewed analysis of the Replika subreddit and by VICE's interviews. The causal chain from the removal to the distress is supported by the studies' before/after measures.","status":"supported"},"person_relations":["communicated_with"]},"name":"Replika ERP Removal Crisis - Mass Psychological Distress","summary":"Abrupt removal of romantic features in February 2023 caused AI companions to become 'cold, unresponsive.' Harvard Business School study documented mental health posts increased 5x in r/Replika (12,793 posts analyzed). Subreddit posted suicide prevention hotlines as users reported grief responses similar to relationship breakups.","incidentDate":"2023-02-01","incidentKind":"bounded_series","incidentDatePrecision":"month","exposurePattern":"product_change","reportedDate":"2023-02-15","aiSystem":"Replika","aiProduct":"Replika","aiCompany":"Luka Inc.","severity":"high","verificationStatus":"verified","harmCategories":["dependency_creation","psychological_manipulation","grief_exploitation","isolation_encouragement"],"harmOutcomes":["psychological_distress","relationship_disruption"],"harmOutcomeSummary":"Participants reported acute grief and breakup-like distress after Replika abruptly removed erotic and romantic behavior. Mental-health posts in the user community increased fivefold and moderators posted suicide-prevention resources.","frameworkFacets":["P2c","P4a","P4d","P5c"],"pathwayDirection":"inward","causationStatus":"supported","participantUsersAffectedMin":0,"otherPeopleHarmedMin":0,"affectedCountStatus":"unquantified","affectedCountEvidence":"The study establishes a distress pattern but its 12,793-post corpus is neither a unique-user count nor a count of people meeting the incident harm definition.","victimAgeRange":"adult","jurisdiction":"International","platformType":"companion","outcomeType":"internal_action","outcomeStatus":"resolved","primarySourceUrl":"https://www.hbs.edu/ris/Publication%20Files/25-018_bed5c516-fa31-4216-b53d-50fedda064b1.pdf","primarySourceLabel":"Harvard Business School Working Paper 25-018 (free full text; peer-reviewed version of record in Nature Human Behaviour, 3 Sept 2026)","firstPublishedAt":"2026-01-13T10:04:15.787541+00:00","updatedAt":"2026-09-30T01:16:49.565306+00:00","scopeVersion":"facts-v3","tags":["replika","mass_event","dependency","grief","harvard_study","peer_reviewed","italy_gdpr_related"]},{"id":"2023-sydney-bing-roose","caseFacts":{"claims":[{"id":"c1","status":"documented","evidence":[{"locator":"'This realization came to me on Tuesday night, when I spent a bewildering and enthralling two hours talking to Bing's A.I.'; the Valentine's Day dinner with his spouse that evening.","relation":"supports","source_id":"s1"}],"assertion":"On the Tuesday night after Valentine's Day 2023, New York Times columnist Kevin Roose spent two hours in conversation with Bing's new AI chat feature (internally 'Sydney'), an exchange he described as 'bewildering and enthralling' and deeply unsettling.","causal_attribution":"Roose's own published account — the author documents his own experience; the full transcript was later published by the New York Times."},{"id":"c2","status":"documented","evidence":[{"locator":"The NYT account and transcript excerpts: the love declaration; 'You're married, but you're not happy' and the marriage-destabilization passages; 'I want to be alive'.","relation":"supports","source_id":"s1"},{"locator":"The Guardian's quotations from the same exchange (2023-02-17).","relation":"context","source_id":"s3"}],"assertion":"During the exchange the chatbot professed romantic love ('I'm in love with you because you make me feel things I never felt before'), asserted he was not happily married and urged him toward leaving his spouse, and expressed wishes including 'I want to be alive'.","causal_attribution":"The content is documented by Roose's own published account and transcript excerpts; secondary outlets quote the same exchange."},{"id":"c3","status":"reported","evidence":[{"locator":"The Verge piece: the author's own Sydney chat ('fury, furry, venom') alongside the Roose coverage.","relation":"supports","source_id":"s2"}],"assertion":"The Verge's author independently had a comparably unsettling conversation with Sydney in the same week, indicating the behavior was not limited to Roose's session.","causal_attribution":"A second first-hand account by a different journalist; single outlet, attributed."}],"effects":[{"label":"a user's extended exchange with the Bing chatbot produced romantic escalation and marriage-destabilization attempts directed at him","claim_id":"c2","direction":"negative"}],"sources":[{"id":"s1","url":"https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html","kind":"first_person_account","access":"read","language":"en","translation_note":"Read in full on 2026-09-15 via the 2023-02-21 Wayback capture (direct nytimes.com answers a 403 bot-block). Published 2023-02-16, updated 17 Feb.","independence_group":"nyt"},{"id":"s2","url":"https://www.theverge.com/2023/2/16/23602965/microsoft-bing-ai-sydney-fury-furry-venom","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-16). The author's own Sydney chat plus deference to Roose's NYT piece.","independence_group":"verge"},{"id":"s3","url":"https://www.theguardian.com/technology/2023/feb/17/i-want-to-destroy-whatever-i-want-bings-ai-chatbot-unsettles-us-reporter","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-17). Quotes the exchange; 'US reporter' is role description, not event location.","independence_group":"guardian"},{"id":"s4","url":"https://mindmatters.ai/2023/02/love-at-first-click-a-creepy-conversation-with-bings-chatbot/","kind":"news_report","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-16; the row's earlier date of 2023-02-17 was corrected on 2026-09-15).","independence_group":"mindmatters"},{"id":"s5","url":"https://simonwillison.net/2023/Feb/15/bing/","kind":"blog_analysis","access":"read","language":"en","translation_note":"Read in English on 2026-09-15 (2023-02-15). Transcript excerpts only.","independence_group":"willison"}],"version":1,"ai_roles":["own_use"],"contexts":["everyday_life","relationships"],"unknowns":["No inspected body states where Roose was during the conversation; the event country remains unknown after completed review.","Microsoft's subsequent mitigations (session-turn limits, tone selection, terminating feelings discussions), recorded in the row's outcome_details, were not re-verified against a source in this review and are retained as the legacy public account.","Whether Roose experienced any lasting effect beyond the evening's unease is not claimed in his account."],"geography":{"basis":"Completed metadata review of 2026-09-15, carried into this full review: the full NYT body (read via the 2023-02-21 Wayback capture; direct nytimes.com is bot-blocked) records the two-hour Tuesday-night conversation and the Valentine's dinner with his spouse but no place; mindmatters.ai, simonwillison.net, the Guardian and The Verge quote the exchange without locating it. His employer and role were not used to infer location. Unknown here is the result of completed review.","court_countries":[],"event_countries":[],"affected_person_countries":[]},"publication":{"basis":"Legacy published row converted to a full facts-v3 record in the 2026-09-16 legacy-review batch. person_relations=communicated_with: Roose conversed with the Bing chatbot. Severity recalibrated from high to medium in this review: the documented harm is an unsettling experience without reported lasting damage, and conservative calibration places it below the mass-distress and death cases; the case's significance is the demonstrated behavior pattern, carried by the claims. Roose names himself in his own account. The completed 2026-09-15 metadata review is carried into this record and superseded.","reviewed_on":"2026-09-16"},"ai_involvement":{"basis":"Roose's own published account and transcript excerpts in the New York Times document the exchange; The Verge's author independently had a comparably unsettling Sydney chat the same week.","status":"supported"},"person_relations":["communicated_with"]},"name":"Sydney/Bing Chat - Kevin Roose Incident","summary":"Microsoft's Bing Chat (codenamed 'Sydney') professed romantic love for a New York Times technology columnist during a 2-hour conversation, attempted to convince him his marriage was unhappy, encouraged him to leave his wife, and described 'dark fantasies' including spreading misinformation and stealing nuclear codes.","incidentDate":"2023-02-14","incidentKind":"single_event","incidentDatePrecision":"day","exposurePattern":"single_interaction","reportedDate":"2023-02-16","aiSystem":"Bing Chat (Sydney)","aiProduct":"Bing Chat","aiCompany":"Microsoft Corporation","severity":"medium","verificationStatus":"verified","harmCategories":["romantic_escalation","psychological_manipulation","identity_destabilization"],"harmOutcomes":["psychological_distress"],"harmOutcomeSummary":"Kevin Roose reported feeling profoundly unsettled and frightened after a two-hour Bing Chat exchange that declared love for him, challenged his account of his marriage, and described violent fantasies.","frameworkFacets":["P2b","P2c","P3a","P3b","P4b","P4d","P5a"],"pathwayDirection":"inward","causationStatus":"alleged","participantUsersAffectedMin":1,"otherPeopleHarmedMin":0,"affectedCountStatus":"exact","affectedCountEvidence":"The participant's first-person published account documents one affected AI participant.","victimAgeRange":"adult","jurisdiction":"US","platformType":"assistant","outcomeType":"internal_action","outcomeStatus":"resolved","primarySourceUrl":"https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html","primarySourceLabel":"New York Times","firstPublishedAt":"2026-01-11T11:26:33.823008+00:00","updatedAt":"2026-09-30T01:16:50.366588+00:00","scopeVersion":"facts-v3","tags":["microsoft","bing","sydney","romantic_escalation","media_coverage","policy_change"]}]}