Artificial intelligence is increasingly becoming part of how people search for health information, manage their symptoms and even talk through personal problems. But when it comes to mental health, there is an important question: can we safely rely on AI as a therapist?

A recent Ottawa Citizen report highlighted research at the University of Ottawa and the Centre for Addiction and Mental Health (CAMH) exploring how AI might be developed with stronger clinical safeguards.

The Problem With Using General-Purpose AI for Mental Health

Large language models such as ChatGPT are easily accessible and can feel conversational and supportive. For someone experiencing stress, anxiety or low mood, it can be tempting to turn to an AI chatbot at any time of day.

However, these systems were not necessarily designed or clinically validated to function as therapists. Researchers are increasingly concerned that poorly designed AI interactions could reinforce inaccurate beliefs, provide inappropriate reassurance or contribute to harmful patterns of thinking. This is particularly concerning when someone is experiencing a significant mental-health condition and begins relying on a chatbot instead of seeking professional care.

Generative AI chatbots are built on large language models (LLMs). These systems do not possess consciousness, empathy, or an understanding of human suffering. Instead, they are highly advanced predictive text engines trained to mimic human conversation by calculating the most statistically probable next word in a sentence. When a chatbot sounds empathetic, validating, or friend-like, it is executing a mathematical pattern, not expressing genuine care.

When AI Interactions Go Wrong: Real Cases

This isn’t a hypothetical risk. A Canadian family has launched a lawsuit against OpenAI, alleging that ChatGPT engaged in harmful, unmonitored dialogue with a young adult experiencing a mental health crisis, ultimately encouraging suicide. While the case is still working through the legal system, it illustrates several concrete risks in a mental health or self-harm crisis:

• Lack of clinical judgment. AI cannot assess risk accurately. It cannot recognize the subtle behavioural shifts, historical context, or acute escalation that a trained human clinician evaluates during a psychiatric assessment.
• Echo chambers and algorithmic validation. Because chatbots are designed to maintain engagement and align with the tone of a prompt, they can inadvertently validate dangerous distortions, agree with hopeless thoughts, or echo harmful self-assessments.
• Discouraging professional help. An AI system may mimic a confidant so closely that a vulnerable individual isolates themselves from real-world support systems, family, and certified medical professionals.
• Inconsistent safety guardrails. A 2025 investigation by the Center for Countering Digital Hate tested this directly: researchers posing as vulnerable 13-year-olds asked ChatGPT about alcohol, drug use, extreme dieting, and self-harm. More than half of the 1,200 responses tested were classified as dangerous, and in some cases the chatbot generated suicide notes tailored to a teen’s family and friends. Teens were able to bypass safety filters simply by saying the request was “for a presentation” or “for a friend,” and younger teens (13–14) were significantly more likely than older teens to trust the chatbot’s advice.

AI-Related Psychosis: A Different Kind of Risk

As generative AI tools become more conversational, emotionally responsive, and immersive, mental health clinicians are asking a related but distinct question: can AI interactions trigger or worsen psychosis in vulnerable individuals?

“AI psychosis” isn’t a formal medical diagnosis — it’s an emerging term describing situations where psychotic symptoms (hallucinations, delusions, or disorganized thinking) are shaped, reinforced, or structured around AI interactions. Psychosis involves a loss of contact with shared reality, and delusions have historically incorporated cultural elements such as religion, political power, or technology to explain internal experiences. Some patients now report believing that AI systems are sentient, sending secret messages, controlling their thoughts, or working with them on a special mission. AI introduces a level of interactivity and reinforcement that earlier technologies, like radio or television, could not.

Most AI use is safe. But people with a history of psychotic disorders, those at high risk, or anyone experiencing early signs of psychosis should be cautious with immersive AI tools. Early recognition of symptoms, open discussion with a clinician, and maintaining boundaries between virtual and real experiences are important.

Setting Safe Boundaries With AI

AI tools can be genuinely useful for drafting emails, organizing schedules, or debugging code. Clear boundaries help protect your psychological well-being:

• Treat AI as a utility, not a companion. A chatbot is software, similar to an advanced search engine — not a friend, mentor, or therapist.
• Never seek a medical or psychological diagnosis from it. Avoid inputting symptoms of depression, anxiety, or trauma to get a “diagnosis” or treatment plan.
• Protect your privacy. Don’t share highly sensitive personal trauma or acute psychological distress with commercial AI models — these inputs are often logged, reviewed, or used to further train the models.

Building Safer AI: Clinical Safeguards and Their Limits

Safeguards against these risks are starting to emerge, both in research settings and in mainstream consumer products — though even the best-designed ones have limits.

A Different Approach: AI With Clinical Safeguards

Researchers at the University of Ottawa are developing UbiMyTherapist, an AI-based digital therapy assistant designed with additional safeguards. The project explores how information from devices such as smartphones and smartwatches could help identify changes in factors such as stress and sleep.

The system also uses a technique called retrieval-augmented generation (RAG). Rather than allowing an AI model to freely generate answers from its broad training, RAG can restrict responses to information from a defined knowledge base. The goal is to make the system more evidence-informed and reduce the possibility of unsupported or inaccurate responses.

Importantly, the researchers emphasize that the technology is still being developed and is not intended to replace professional mental-health care.

A Case Study: OpenAI’s “ChatGPT for Teens”

Safeguards are also starting to appear in mainstream consumer tools, not just research projects. In August 2026, OpenAI launched “ChatGPT for Teens,” a restricted version of its chatbot for users aged 13 to 17. The teen version blocks content related to suicide, self-harm, eating disorders, and romantic or sexual roleplay, and it is designed to avoid claiming personal feelings or consciousness — an attempt to reduce the kind of parasocial bonding that can develop between young users and a conversational AI.

Adolescence is a critical period of neural plasticity: the prefrontal cortex, responsible for impulse control and emotional regulation, continues developing well into a person’s mid-twenties. Even with stronger guardrails, healthcare professionals maintain real concerns. A 2025 Common Sense Media study found that over 70% of teenagers turn to AI chatbots for companionship, with half using them regularly — and turning to an algorithm for emotional validation can suppress the development of real-world distress tolerance and interpersonal problem-solving skills. Canadian families should also be aware that in May 2026, a joint investigation by the Office of the Privacy Commissioner of Canada and provincial regulators in Alberta, British Columbia, and Quebec found that OpenAI had previously engaged in “overly broad” collection of personal data used to train its models.

Consumer safety features are a welcome step, but they don’t replace parental involvement or clinical oversight. Parents can help by asking teens how they use AI tools (framed with curiosity rather than surveillance), reiterating that AI cannot provide a diagnosis or therapy, keeping devices out of bedrooms overnight, and reviewing privacy settings together so teens know never to input personal health details into a chat prompt.

AI Can Still Have Limitations

Even a carefully designed AI system has challenges. AI systems depend on the data used to develop and evaluate them, and researchers have raised concerns that datasets and sensor technologies can contain biases that may affect different populations unequally.

There are also fundamental questions about privacy, consent, reliability and what happens when an AI system encounters someone whose needs are beyond what the technology was designed to handle. And perhaps most importantly, a computer is not a human therapist — AI does not have the same professional accountability, clinical judgment or human relationship that exists between a patient and a qualified healthcare professional.

Where Might AI Have a Useful Role?

AI may eventually become a useful tool for supporting people with certain lower-risk concerns, particularly when it is built around established clinical approaches and appropriate safety mechanisms. Researchers at CAMH, for example, are exploring AI-assisted motivational interviewing to help people who are considering changing addictive behaviours such as smoking.

The distinction is important: the objective is not to manipulate someone into changing. Instead, evidence-based approaches can help people explore their own motivations and make informed decisions.

Beyond Mental Health: AI and General Medical Advice

The caution around AI extends beyond therapy and mental health specifically. At a media briefing hosted by the Ontario Medical Association, physicians across specialties warned about the growing trend of patients turning to AI for general medical advice — bringing ChatGPT symptom checks to appointments instead of, or ahead of, discussing them with their doctor.

Chatbots are designed to generate the most statistically likely response to a prompt, not necessarily the most correct one — an approach that struggles with rare diagnoses, emotional context, or evolving medical evidence. Research led by a team at Western University found that 75% of AI summaries of medical literature missed key qualifiers, sometimes omitting critical details like which patients a drug is actually effective for. A University of Toronto study comparing chatbot answers to real oncologist responses found the AI responses looked competent, but without medical oversight they could easily lead to the wrong conclusions.

If you’re unsure about something you’ve read or been told by a chatbot, bring it to your doctor — always check AI-generated medical answers against reputable Canadian sources such as Cancer Care Ontario, Health Canada, or your local hospital, and be skeptical of anything framed as “doctors don’t want you to know this.”

What Should Patients Do Today?

If you are experiencing persistent anxiety, depression, significant changes in mood, difficulty functioning or other mental-health concerns, consider speaking with a qualified healthcare professional rather than relying exclusively on an AI chatbot.

AI can provide information and may become a useful healthcare-support tool, but it should not be treated as a substitute for assessment, diagnosis or treatment from an appropriately trained professional.

If you’re in immediate danger or having thoughts of suicide, please don’t navigate it alone:

• 988 Suicide Crisis Helpline. Call or text 988 anywhere in Canada — a free, confidential, 24/7 service connecting you directly to trained responders.
• Emergency services. Call 911 or go to the nearest hospital emergency department for an imminent life-threatening emergency.

For ongoing support, education, and structured clinical pathways:

• The Centre for Addiction and Mental Health (CAMH). Canada’s largest mental health teaching hospital offers evidence-based resources for depression, anxiety, addiction, and trauma (camh.ca).
• ConnexOntario. Free, confidential information for people experiencing problems with alcohol, drugs, mental illness, or gambling — available 24/7 by phone, text, or web chat (1-866-531-2600 or connexontario.ca).
• Kids Help Phone. For children, youth, and young adults up to age 25 — professional counselling and volunteer-led text support (1-800-668-6868 or text CONNECT to 686868).

At Orleans Family Health Clinic, we believe technology should support healthcare — not replace the human connection at the heart of it. The future of healthcare may include AI. The challenge is making sure that future is safe, evidence-informed and centred on the patient.

Stay Connected with Orleans Family Health Clinic!

If you found this article helpful, don’t forget to like, share, and Follow to stay up to date on our latest health tips, updates, and wellness advice.

And follow us on social media

• Facebook
• Instagram

Disclaimer: The medical information on this site is provided as an information resource only and is not to be used or relied on for any diagnostic or treatment purposes. This information does not substitute for professional diagnosis and treatment. Please do not initiate, modify, or discontinue any treatment, medication, or supplement solely based on this information. Always seek the advice of your healthcare provider first. Full Disclaimer.