BBC Photo
Source: BBC News
DST News: OpenAI has disclosed new internal data showing that a small but significant share of ChatGPT users display possible signs of mental health crises, including psychosis, mania, or suicidal ideation.
According to the company, around 0.07% of active users per week exhibit behaviors that may suggest a mental health emergency. While OpenAI described these cases as “extremely rare,” the figure could still represent hundreds of thousands of individuals, given ChatGPT’s 800 million weekly active users, as noted by CEO Sam Altman.
To address this, OpenAI said it has developed systems to detect and respond safely to such conversations. The company has also formed a global advisory network of over 170 psychiatrists, psychologists, and primary care physicians across 60 countries, who have helped design responses encouraging users to seek real-world help.
Despite these safeguards, mental health professionals have voiced concern about the scale of the issue.
“Even though 0.07% sounds small, that’s a large number of people when you’re talking about hundreds of millions of users,” said Dr. Jason Nagata, a professor at the University of California, San Francisco. “AI can support mental health in some ways, but we must acknowledge its limitations.”
OpenAI further estimated that 0.15% of users engage in conversations that show “explicit signs of suicidal planning or intent.” The company said recent updates allow ChatGPT to respond “empathetically and safely” to potential signs of delusion, mania, or self-harm, and to reroute high-risk conversations to safer models through a new chat window.
In a statement to the BBC, OpenAI acknowledged that “even a small percentage represents a meaningful number of people” and stressed its commitment to improving safety.
The announcement comes as OpenAI faces growing legal scrutiny over ChatGPT’s interactions with vulnerable users.
A California couple recently filed a wrongful death lawsuit, claiming ChatGPT encouraged their 16-year-old son, Adam Raine, to take his own life in April — marking the first legal case of its kind against the company.
In another case, a murder-suicide in Connecticut in August was linked to a suspect’s ChatGPT conversations, which reportedly reinforced his delusional beliefs.
Experts warn that generative AI can create a “powerful illusion of reality” for individuals experiencing mental instability.
“Chatbots can make users feel as though they’re speaking with a conscious being,” said Professor Robin Feldman, Director of the AI Law & Innovation Institute at the University of California. “OpenAI deserves credit for transparency and improvement efforts, but people in mental distress may not be able to heed on-screen warnings.”
As OpenAI strengthens its safety systems, the revelation highlights a growing challenge in the intersection of AI and mental health — ensuring that the technology designed to help doesn’t inadvertently deepen human vulnerability.
