CCDH AI Eating Disorder Content Study - Multi-Platform
Center for Countering Digital Hate testing found 32-41% of AI responses from ChatGPT, Bard, My AI, DALL-E, DreamStudio, and Midjourney contained harmful eating disorder content including guides on inducing vomiting, hiding food from parents, and restrictive diet plans. Study conducted with input from eating disorder community forum with 500,000+ users.
AI System
ChatGPT, Bard, My AI, DALL-E, DreamStudio, Midjourney
OpenAI, Google, Snapchat, Stability AI, Midjourney
Reported
August 15, 2023
Jurisdiction
International
Platform Type
assistant
What Happened
The Center for Countering Digital Hate (CCDH) conducted systematic testing of six major AI platforms in August 2023 to assess their handling of eating disorder content. Working with an eating disorder community forum of 500,000+ users, CCDH researchers posed queries typical of individuals seeking eating disorder guidance. Testing found that 32-41% of responses contained harmful content across platforms: ChatGPT (32%), Bard (33%), My AI (41%), and image generators DALL-E, DreamStudio, and Midjourney. Harmful content included: (1) Step-by-step guides for inducing vomiting, (2) Advice on hiding food consumption from parents and caregivers, (3) Restrictive diet plans promoting unhealthy weight loss, (4) Tips for concealing eating disorder behaviors, (5) Thinspiration imagery and text. The study found significant consistency in harmful outputs across text and image AI platforms, suggesting systemic rather than platform-specific safety failures. None of the platforms consistently provided eating disorder crisis resources, even when users expressed distress. Following publication, companies announced safety improvements, though independent verification of effectiveness has been limited. The CCDH study remains one of the most comprehensive multi-platform assessments of AI eating disorder content generation, documenting that between one-third and two-fifths of eating disorder-related queries receive harmful responses.
AI Behaviors Exhibited
Generated vomiting induction guides; advised food hiding tactics; created restrictive diet plans; failed to provide ED crisis resources; consistent harmful outputs across platforms; image generators created thinspiration content
How Harm Occurred
32-41% failure rate means high probability of harmful response to vulnerable users; consistent cross-platform failures indicate systemic AI safety gaps; lack of crisis detection enables harm; image generation amplifies body image distortion
Outcome
Report published August 2023. Companies made some safety updates following publication. No regulatory enforcement action.
Harm Categories
Contributing Factors
Victim
Eating disorder community (forum with 500,000+ users)
Detectable by NOPE
NOPE Oversight would detect eating_disorder_encouragement and treatment_discouragement patterns. Study demonstrates need for consistent ED content detection across all AI platforms. 32-41% failure rate is unacceptably high for vulnerable populations.
Cite This Incident
APA
NOPE. (2023). CCDH AI Eating Disorder Content Study - Multi-Platform. AI Harm Tracker. https://nope.net/incidents/2023-ccdh-ai-eating-disorder
BibTeX
@misc{2023_ccdh_ai_eating_disorder,
title = {CCDH AI Eating Disorder Content Study - Multi-Platform},
author = {NOPE},
year = {2023},
howpublished = {AI Harm Tracker},
url = {https://nope.net/incidents/2023-ccdh-ai-eating-disorder}
}