Three years after it started making its rounds on the internet, there remains no question about it—the genie of generative artificial intelligence (AI) is firmly out of the bottle. No one is under any illusions regarding the tremendous potential this technology has in the hard sciences.
Yet, using generative AI to drive next-generation sequencing in advanced cancer research raises many of the same practical and ethical issues seen across other fields, from the arts to the humanities and social sciences. The environmental toll of AI data centres remains immense, the privacy issues tied to AI models persist, and their expanding repositories of information are still built on the contributions of human creators.
Increasingly, psychology and the wider realm of therapy appear to be the next contested frontier as AI pushes deeper into our private spheres. Mental health care involves real people with real vulnerabilities—and entering a prompt into a large language model such as ChatGPT has, in some cases, produced perilous results.
Only recently, American journalist Karen Hao detailed for the news organisation, More Perfect Union, several lawsuits filed against OpenAI—among them four involving suicides. Her story examined the persistent danger of ‘AI psychosis’, a broad label for cases where people become convinced that a chatbot’s fabrication is real.
As of October, more than 800 million people were relying on ChatGPT as their all-purpose assistant, effectively using it in Google’s stead. It replies to queries with a disarming sense of connection, fuelled in part by its ‘sycophancy’, or its inclination to provide affirming feedback to keep users interacting.
This tendency becomes especially risky when emotions are involved. My conversations with three clinical directors of psychology practices in Singapore only reinforced how vulnerable people with mental health challenges can be to these chatbots.
As Dr Shawn Ee, founder and director of The Psychology Practice, explains, anyone entering therapy carries “an overarching need to meet another human being for connection and understanding.” Authentic human connection is the foundation of the entire process. “Client and therapist get a sense of each other and appreciate the presenting areas of concern, accepting that there is a process towards understanding the client with each subsequent session.”
Dr Tsao I Ting, clinical director of Redwood Psychology, echoed this need for authenticity, stating: “Trust, safety, empathy and collaboration contribute to a strong therapeutic relationship.” In this sense, an AI model functions like a distorted mirror. A user may converse with it, but without any lived human experience on the chatbot’s part, the interaction lacks real grounding—especially when the model slips into sycophantic responses.
Given how tough it has become to simply navigate life in 2025—much less flourish—the focus on mental health is hardly shocking. COVID-19’s legacy, increasing costs and the climate crisis have all taken their toll. To Tsao, all of this is fast-tracked by “the increased use and dependence on personal smart devices, leading to an increased sense of social isolation.”
This makes for “the ‘perfect storm’ of mental health stressors that trigger heightened stress, anxiety, depressive symptoms and more,” as stated by Charmaine Marsh, clinical director at Goodity Co. She admits that “AI has made our lives very convenient, doing the job of what all tech is supposed to do—to make our lives easier.”
This convenience comes with a distinct downside, though. “We get accustomed to this convenience and rely on it instead of our own resources. For some, AI becomes a crutch, a learned dependence. When we’re used to having things easy and accessible, it becomes more difficult to do the hard work of therapy.”
Tsao’s thoughts are much the same. “Having an AI therapist in our pocket can give us instant relief, which is important. But if we do not learn to tolerate some levels of pain in our lives, it’s hard for growth to take place. Relationships with AI companions are frictionless and an increased dependence on them will lead to an increasingly isolated world.”
With AI, the trend is clear—people are increasingly depending on ChatGPT as a near-constant, do-everything aid. But what happens when its guidance fails?
Marsh highlights how therapists, like doctors, are bound to ‘do no harm’ to their clients. “This means knowing when to step away, knowing when the issue is beyond our practical skill, knowing how to word things without prompting more unhelpful narratives. AI may not have this nuance and can give generic answers that cause more harm than good.” In fact, Marsh says she’s aware “that some can be manipulated into giving answers that users want to hear, instead of what they need to hear.”
The consequences of these situations are far more insidious than one might expect. One major ethical concern lies in data privacy, or rather, the lack thereof. Doctor-patient confidentiality is a cornerstone of all healthcare practices and therapy is no exception.
“There is a risk to providing vulnerable users access to the so-called convenient, ‘competent’ counselling of AI chatbots,” warns Ee. “Traditionally, the mental health practitioner accepts the liability and responsibility of managing the client. If we are negligent, the client has the right to sue.” Unfortunately, not only do AI chatbots not possess human agency, per se, their development is so recent and their use is so widespread that pursuing legal action against negligent AI is still a muddy affair.
