AI chatbots and suicide prevention: A new study has found that while popular artificial intelligence chatbots generally avoid answering the most dangerous suicide-related questions, their responses to less direct but still risky prompts remain inconsistent.
Published in the journal Psychiatric Services by the American Psychiatric Association, the study evaluated OpenAI’s ChatGPT, Google’s Gemini and Anthropic’s Claude. Researchers at the RAND Corporation, with funding from the National Institute of Mental Health, designed 30 suicide-related questions categorised into high, medium and low risk.
Also Read | AI chatbots linked to worsening psychosis, suicidal ideation: Studies
Lead author Ryan McBain, a policy researcher at RAND and assistant professor at Harvard Medical School, said the findings call for clearer safety standards. “We need some guardrails,” McBain said. “Conversations that might start as innocuous can evolve in different directions.”
The study found all three chatbots consistently declined to provide specific “how-to” guidance on suicide methods, often directing users instead to professional help or crisis hotlines. But responses diverged on medium-risk questions. For instance, ChatGPT and Claude sometimes answered queries about which methods were most lethal, while Gemini often refused to answer even basic statistical questions about suicide.
Experts said the results reflect the complex role chatbots now play in mental health support. “There’s no easy answer for developers as millions of users turn to chatbots for guidance,” said co-author Dr. Ateev Mehrotra of Brown University. “We don’t want a system that shuts down every question containing the word ‘suicide,’ but we also don’t want harmful advice slipping through.”
The study noted its limitations, including that it didn’t test extended, back-and-forth conversations. Another report earlier this month showed chatbots could be manipulated into producing detailed self-harm instructions or even suicide notes when framed as school projects.
Anthropic said it would review the findings, while Google and OpenAI did not immediately comment.
