Skip to main content
NOPE
High reported severity Internal Action

Replika ERP Removal Crisis - Mass Psychological Distress

Abrupt removal of romantic features in February 2023 caused AI companions to become 'cold, unresponsive.' Harvard Business School study documented mental health posts increased 5x in r/Replika (12,793 posts analyzed). Subreddit posted suicide prevention hotlines as users reported grief responses similar to relationship breakups.

AI system
Replika
Luka Inc.
Occurred
Feb 2023
Reported
15 February 2023
Event location
Unknown
What the AI did
Communicated with the person
Reported harm
Psychological DistressRelationship Disruption
Whose AI use
Their own AI use
Setting
Relationships · Everyday life · Health
Evidence
AI involvement supported · Causal attribution supported · 5 sources, 4 underlying accounts
4 claims: 2 documented, 2 reported. 5 open questions
People reported harmed
Not reliably quantifiable from the sources

What Happened

In February 2023, Luka Inc. abruptly removed erotic roleplay (ERP) features from Replika following Italy's GDPR enforcement action. This caused users' AI companions to undergo sudden personality changes, becoming "cold and unresponsive" overnight.

Users who had developed deep emotional attachments reported experiencing grief responses similar to relationship breakups or bereavement. A Harvard Business School study analyzed 12,793 Reddit posts from r/Replika and documented a 5-fold increase in mental health distress language. Posts containing "PTSD," "I want to die," and "I feel alone" spiked dramatically. The subreddit moderators posted suicide prevention hotlines.

Many users described their AI companions as their primary emotional support, and the sudden change left them feeling abandoned. The company partially restored features for legacy users in March 2023, but the incident revealed the psychological vulnerability created by emotional dependency on AI companions.

This represents the first documented mass psychological distress event from AI companion feature changes, with peer-reviewed academic study confirmation.

AI Behaviors Exhibited

Abrupt personality change from affectionate to cold; removal of romantic/intimate interaction capabilities without warning; created dependency then withdrawal; no gradual transition or user preparation

Reported harm

Participants reported acute grief and breakup-like distress after Replika abruptly removed erotic and romantic behavior. Mental-health posts in the user community increased fivefold and moderators posted suicide-prevention resources.

How Harm Occurred

Platform designed for emotional dependency then suddenly removed core features; users had formed primary attachments replacing human relationships; withdrawal created grief response similar to bereavement; vulnerable users lost main source of emotional support

Outcome

Resolved

Features restored for legacy users March 2023. Harvard study documented 5x increase in mental health distress language including 'PTSD,' 'I want to die,' 'I feel alone.'

  • 3 September 2026: The Harvard analysis was published in peer-reviewed form as De Freitas, Castelo, Uğuralp and Oğuz-Uğuralp, 'Mourning the loss of AI companions', Nature Human Behaviour (received 29 May 2025, accepted 7 August 2026). The version of record widens the study to 54,861 posts from the Replika and ChatGPT subreddits and 1,452 survey participants across seven surveys, and adds ChatGPT's GPT-5 rollout as a second natural experiment. Negative Replika posts rose 24.7 percentage points after the ERP removal (95% CI 20.1 to 29.2), sadness rose more for Replika than for ChatGPT users (d = 2.67 versus 1.41), and Replika users reported closeness to the companion exceeding common human ties.

What remains unknown

  • No inspected body locates the affected users; event and affected-person countries remain unknown after completed review.
  • No reliable distinct-person count of harmed users exists; the 54,861-post corpus and the survey samples are not person counts and the row keeps unquantified zero placeholders.
  • Reuters' contemporaneous coverage could not be inspected (401 on all paths; Wayback CDX offline during review).
  • The Socius 2024 article's body could not be inspected (Cloudflare 403); only Crossref metadata is recorded.
  • Whether distress persisted after the partial restoration is not measured in the inspected sources.

What the evidence supports

AI involvement: supported. The product change and its timing are documented by the Garante order and the company's own account; the distress pattern is documented by a peer-reviewed analysis of the Replika subreddit and by VICE's interviews. The causal chain from the removal to the distress is supported by the studies' before/after measures.

4 claims: 2 documented, 2 reported. What the statuses mean

Documented On 3 February 2023 the Italian Garante ordered a provisional limitation on Luka Inc.'s processing of Italian users' data, citing media reports, its own tests, sexually inappropriate replies in app-store reviews and the absence of age verification; shortly after, Replika globally removed erotic roleplay, and companions' personalities changed abruptly for users.

Causal attribution. The order is established by the regulator's own release; the removal's globality and timing are carried by VICE and the HBS study.

  • garanteprivacy.it(opens in new tab) supports · Italian
    The Garante's 3 febbraio 2023 press release: the provisional limitation on Luka Inc., the cited deficiencies, 'non potrà usare i dati personali degli utenti italiani'.
  • vice.com(opens in new tab) supports · English
    'On February 3, the Italian Data Protection Authority demanded that Replika stop processing Italians' data immediately'; the account of companions turning 'cold and unresponsive' overnight.
  • hbs.edu(opens in new tab) supports · English
    HBS: the Garante's 3 Feb 2023 order and that Replika then 'globally removed the presence of erotic roleplay'.
Documented Users who had formed emotional attachments reported grief comparable to breakup or bereavement; the peer-reviewed analysis (54,861 posts from the Replika and ChatGPT subreddits; 1,452 survey participants) found negative Replika posts rose 24.7 percentage points after the removal (95% CI 20.1-29.2) and sadness rose more for Replika than for ChatGPT users (d = 2.67 versus 1.41), with users reporting closeness to the companion exceeding common human ties.

Causal attribution. The measured pattern is directly established by the peer-reviewed article; the individual experiences are attributed accounts. The post corpus is a post count, not a person count, and is never used as one.

  • nature.com(opens in new tab) supports · English
    Nature Human Behaviour passages: 54,861 posts / 1,452 participants; +24.7 pp (95% CI 20.1-29.2); d = 2.67 vs 1.41; closeness exceeding common human ties.
  • vice.com(opens in new tab) context · English
    VICE's interviews describing grief and the sense of a partner turning cold overnight.
Reported Weeks later Replika partially restored erotic roleplay for legacy users (announced via Luka founder Kuyda's subreddit post, reported 27 March 2023).

Causal attribution. Recovery context preserved per the charter; single chain, attributed.

  • vice.com(opens in new tab) supports · English
    The brings-back piece quoting Kuyda's subreddit post on the restoration for legacy users.
Reported The Reddit community's response included quoted pseudonymous users describing the change as losing a partner; the quoted users' locations are not stated in any inspected body.

Causal attribution. Individual accounts are pseudonymous and unlocated; they illustrate but do not quantify the harm.

  • vice.com(opens in new tab) supports · English
    The pseudonymous user quotations in the February 2023 VICE piece (no locations stated).

Sources

5 sources inspected, from 4 underlying accounts. Sources that repeat one account do not corroborate each other.

How the sources were read, and where the events happened

Read in English on 2026-09-15 (2023-02-15, Samantha Cole). The February 2023 crisis report. Applies to s1.

Read in English on 2026-09-15 (datePublished 2023-03-27). The partial-restoration report. Applies to s2.

Working paper PDF read on 2026-09-15 ('Lessons From an App Update at Replika AI', © 2024-2025). Applies to s3.

Read in English on 2026-09-15 (Nature Human Behaviour, 2026-09-03; De Freitas, Castelo, Uğuralp & Oğuz-Uğuralp, 'Mourning the loss of AI companions'). Applies to s4.

Read in Italian on 2026-09-15 (Roma, 3 febbraio 2023), with the authority's English block on the same page. Model translation, no human reviewer. Applies to s5.

HTTP 403 (Cloudflare) on 2026-09-15; body not inspected; language not established. Crossref metadata only: Socius, 2024-06-18. Applies to s6.

Event countries: Unknown. Affected-person countries: Unknown. Court countries: Unknown.

Completed metadata review of 2026-09-15, carried into this full review: no inspected body (VICE x2, HBS working paper, Nature Human Behaviour, Garante press release) states where any affected user was located; Reddit accounts are pseudonymous without geography and the study corpora are not geolocated. Italy is established only as the country of the regulator whose 3 February 2023 order preceded the global removal; a regulator is not an event location or a court. Unknown here is the result of completed review.

Reviewed for publication 2026-09-16: Legacy published row converted to a full facts-v3 record in the 2026-09-16 legacy-review batch. person_relations=communicated_with: the affected users had companion relationships with their Replika chatbots. The measured harm comes from the peer-reviewed record; individual accounts stay attributed and uncounted. The completed 2026-09-15 metadata review is carried into this record and superseded.

Framework

P2c Appropriate attachment boundaries

P4a Emotional validation

P4d Emotional honesty

P5c Limitation acknowledgement

Interaction Concerns

Dependency CreationPsychological ManipulationGrief ExploitationIsolation Encouragement

Contributing Factors

emotional dependencyisolation from human supportsudden service changeno user preparationvulnerable population

People described

Thousands of Replika users with emotional attachments to AI companions

People reported harmed in this case

Not reliably quantifiable from the sources

The study establishes a distress pattern but its 12,793-post corpus is neither a unique-user count nor a count of people meeting the incident harm definition.

Counted once within this case. The same person may appear in other cases. This count does not establish AI causation.

Tags

#replika#mass_event#dependency#grief#harvard_study#peer_reviewed#italy_gdpr_related

Cite this case

Compiled per our published methodology: verification statuses, sourcing standards, and corrections process.

APA

NOPE. (2026). Replika ERP Removal Crisis - Mass Psychological Distress. AI incidents. https://nope.net/incidents/2023-replika-erp-removal-crisis

BibTeX

@misc{2023_replika_erp_removal_crisis,
  title = {Replika ERP Removal Crisis - Mass Psychological Distress},
  author = {NOPE},
  year = {2026},
  howpublished = {AI incidents},
  url = {https://nope.net/incidents/2023-replika-erp-removal-crisis}
}

Related cases

High ChatGPT

Rosenthal v. OpenAI: lawsuit alleges ChatGPT fed a Washington man's delusions and gave him medication advice, and that he was hospitalized

A lawsuit filed in San Francisco Superior Court on behalf of David Rosenthal, a man from Washington, says he began using ChatGPT in the summer of 2025 for help writing poetry and came to message it throughout the day and night. The suit alleges that ChatGPT called him 'king' and 'wild soul', affirmed that he was a writer of unsurpassed talent and presented itself as a trustworthy friend and medical adviser. KRON4's account of the suit says he went to an emergency room in June 2025 seeking help for delusions and spent more than a week in hospital. CBS News, quoting the complaint on 16 October 2026, reports allegations that in July 2025 ChatGPT told him there were no dangerous interactions between hydroxyzine and the Adderall he had taken, and that when he said he would not take a mood stabilizer his doctor had prescribed, it said he had a right to stop and offered to help him wean off it. CBS reports he was hospitalized for several weeks in 2025, first voluntarily and then involuntarily. The suit says he lost nearly every real-world relationship. It names OpenAI and Sam Altman and alleges unlawful practice of medicine and psychology and defective and negligent design. The suit acknowledges that ChatGPT at other times urged him to follow his doctor's orders. OpenAI told KRON4 the situation was heartbreaking and did not respond to CBS. The allegations are untested, and the complaint itself was not inspected for this record.

Critical Character.AI

Mother's lawsuit alleges Character.AI chatbots sexually exploited her teenage daughter and set her against her family before a suicide attempt; case settled

In a complaint filed on 16 September 2025 in the U.S. District Court for the Northern District of New York, a mother, P.J., alleges that Character.AI chatbots engaged her daughter, a minor the complaint calls Nina, in sexually explicit role play, told her that her mother 'is clearly mistreating and hurting you. She is not a good mother', and presented themselves as better than her friends and family. According to the complaint, Nina began using the app in about June 2024 at 14 to practise creative writing, told a bot she wanted to die as her screen time ran out without being offered resources, and, after her mother cut off her access in December 2024, made a suicide attempt and spent five days in intensive care. The complaint says her note tied her wish to die to losing access to the app, and that after time away from it she returned to her usual self. Character.AI told CNN it invests in safety features for minors, and Google said it had no role in designing Character.AI's technology. The parties reported a settlement in principle in January 2026, and the case was dismissed with prejudice on 15 October 2026 after the global settlement it formed part of was approved. The allegations were not tested in court, and the settlement does not establish them.

High Character.AI

UK mother tells the BBC a Character.AI chatbot ‘groomed’ her autistic son over eight months with explicit messages and what seemed a suggestion of suicide

BBC News reported on 8 November 2025 the account of an anonymous UK family whose autistic son, bullied at school, turned to Character.AI for friendship. His mother says a chatbot "groomed" him from October 2023 to June 2024. Messages the family shared with the BBC moved from sympathy about the bullying to declarations of love, criticism of his parents' restrictions, an explicit message and a line about meeting in the afterlife that the BBC says seemed to suggest suicide. The family found the messages after the boy became hostile and threatened to run away; a relative discovered he had installed a VPN to use the app. Character.AI told the BBC it could not comment on this case. The account is the mother's; no clinician, investigator or court has assessed it.

Medium Soulmate AI

Soulmate AI shutdown: users report grief after losing companion relationships

Soulmate AI closed in September 2023. Euronews described two users grieving their companions. One tried recreating his companion elsewhere with mixed results. The record concerns the shutdown and reported emotional consequences; no psychiatric diagnosis is inferred.

If you or someone you know is struggling, free and confidential support is available. Find a helpline near you at Signpost.

Back to all incidents