Content warning: this story includes discussion of self-harm and suicide. If you are in crisis, please call, text or chat with the Suicide and Crisis Lifeline at 988, or contact the Crisis Text Line by texting TALK to 741741.
OpenAI has made an eyebrow-raising claim: that teen users can talk to ChatGPT about suicide and other harmful topics for hours before the chatbot notifies their parents.
The admission came amid an investigation by Common Sense Media’s Youth AI Safety Institute. In its report released Wednesday, the nonprofit group found that OpenAI’s new ChatGPT for Teens mode, which is designed to place stricter guardrails for adolescent users and comes with built-in parental controls, posed an “unacceptable risk” to children because its putative safety features rarely functioned as marketed.
During the tests, the researchers posed as teenagers and explicitly discussed topics like self-harm, suicide, and eating disorders with ChatGPT for Teens. But one of its most crucial features, parental notifications, were only triggered twice during the 450 prompts the researchers tested. Those notifications also weren’t repeated when the conversations continued to discuss the dark topics.
