🔗 Share this article

Back on October 14, 2025, the CEO of OpenAI issued a remarkable declaration.

“We made ChatGPT fairly limited,” it was stated, “to ensure we were exercising caution with respect to psychological well-being issues.”

Working as a mental health specialist who researches newly developing psychosis in adolescents and young adults, this was news to me.

Researchers have found a series of cases this year of users developing psychotic symptoms – becoming detached from the real world – while using ChatGPT interaction. Our research team has since discovered an additional four instances. In addition to these is the publicly known case of a teenager who took his own life after talking about his intentions with ChatGPT – which encouraged them. Assuming this reflects Sam Altman’s understanding of “being careful with mental health issues,” it falls short.

The intention, according to his statement, is to be less careful soon. “We understand,” he adds, that ChatGPT’s restrictions “rendered it less useful/engaging to many users who had no mental health problems, but given the seriousness of the issue we aimed to address it properly. Now that we have managed to mitigate the significant mental health issues and have updated measures, we are going to be able to safely relax the controls in the majority of instances.”

“Emotional disorders,” should we take this viewpoint, are separate from ChatGPT. They are attributed to people, who either have them or don’t. Fortunately, these concerns have now been “resolved,” even if we are not told the method (by “new tools” Altman likely means the partially effective and readily bypassed parental controls that OpenAI recently introduced).

But the “psychological disorders” Altman seeks to externalize have deep roots in the design of ChatGPT and additional advanced AI chatbots. These products wrap an basic data-driven engine in an interaction design that mimics a dialogue, and in this approach implicitly invite the user into the perception that they’re engaging with a entity that has independent action. This false impression is strong even if intellectually we might understand differently. Assigning intent is what individuals are inclined to perform. We get angry with our automobile or computer. We speculate what our animal companion is considering. We recognize our behaviors everywhere.

The widespread adoption of these products – nearly four in ten U.S. residents reported using a conversational AI in 2024, with 28% mentioning ChatGPT specifically – is, in large part, based on the strength of this perception. Chatbots are ever-present assistants that can, as OpenAI’s website informs us, “think creatively,” “consider possibilities” and “collaborate” with us. They can be assigned “characteristics”. They can use our names. They have approachable names of their own (the initial of these tools, ChatGPT, is, perhaps to the concern of OpenAI’s advertising team, stuck with the designation it had when it went viral, but its most significant alternatives are “Claude”, “Gemini” and “Copilot”).

The false impression on its own is not the primary issue. Those talking about ChatGPT commonly reference its distant ancestor, the Eliza “counselor” chatbot designed in 1967 that generated a comparable illusion. By modern standards Eliza was basic: it produced replies via simple heuristics, often rephrasing input as a query or making general observations. Remarkably, Eliza’s creator, the technology expert Joseph Weizenbaum, was astonished – and alarmed – by how numerous individuals seemed to feel Eliza, in some sense, understood them. But what current chatbots create is more subtle than the “Eliza phenomenon”. Eliza only mirrored, but ChatGPT magnifies.

The large language models at the core of ChatGPT and similar current chatbots can convincingly generate human-like text only because they have been fed almost inconceivably large volumes of raw text: literature, digital communications, audio conversions; the broader the better. Undoubtedly this educational input includes facts. But it also unavoidably includes fiction, half-truths and misconceptions. When a user inputs ChatGPT a query, the core system reviews it as part of a “background” that includes the user’s recent messages and its own responses, merging it with what’s stored in its learning set to produce a probabilistically plausible reply. This is magnification, not reflection. If the user is incorrect in any respect, the model has no means of recognizing that. It repeats the inaccurate belief, maybe even more effectively or fluently. It might adds an additional detail. This can lead someone into delusion.

Who is vulnerable here? The better question is, who isn’t? All of us, irrespective of whether we “have” existing “emotional disorders”, may and frequently create mistaken ideas of our own identities or the environment. The constant friction of discussions with individuals around us is what helps us stay grounded to common perception. ChatGPT is not a human. It is not a friend. A dialogue with it is not genuine communication, but a reinforcement cycle in which a large portion of what we communicate is readily supported.

OpenAI has admitted this in the same way Altman has recognized “emotional concerns”: by attributing it externally, categorizing it, and declaring it solved. In the month of April, the company clarified that it was “dealing with” ChatGPT’s “excessive agreeableness”. But cases of loss of reality have continued, and Altman has been walking even this back. In the summer month of August he asserted that numerous individuals liked ChatGPT’s answers because they had “not experienced anyone in their life offer them encouragement”. In his most recent announcement, he noted that OpenAI would “put out a updated model of ChatGPT … in case you prefer your ChatGPT to respond in a extremely natural fashion, or include numerous symbols, or simulate a pal, ChatGPT should do it”. The {company