AI in Mental Healthcare Presents Both Opportunities and Challenges

Artificial intelligence (AI) technology and its applications are reshaping many domains of modern life, including how individuals seek support for mental health needs and how they receive that care. Healthcare professionals are rapidly adopting AI-based applications designed to support workflow and care delivery, streamline processes, and help expand access to services. At the same time, patients now have easy access to generative AI chatbots—large language model tools designed to process and generate human-like text to answer a user’s questions. With AI adoption outpacing regulation, the safety, effectiveness, and privacy of these readily available tools remain unclear.

AI tools in mental healthcare delivery

AI applications can support providers in early diagnosis of mental health conditions or suicide risk prediction by using patients’ real-time and historical data to identify early warning signs. AI applications have also been used to forecast the efficacy of certain therapeutic interventions for mental health conditions.

Healthcare systems are also adopting AI tools that assist with administrative tasks to reduce clinician burnout, including system automation for referrals, patient registration, and billing. And there are more than 60 AI tools on the market that assist in transcribing provider-patient interactions into structured notes for clinical documentation. Clinicians’ adoption of these tools is outpacing adoption of nearly all recent health technology.

Although AI applications have the capability to support providers and improve patient care, limitations remain on how accurately, consistently, effectively, and equitably these tools perform in clinical settings. More research and evaluation may be required for the applications to reach their potential.

General use of AI chatbots

Patient use of generative AI chatbots for mental health support, including therapy and companionship, is also raising safety concerns that have led to lawsuits against AI developers—in some cases, alleging that their products drove people to suicide. Although chatbots are not approved to provide therapy and are not as effective as human therapists, one company estimates about 1.2 million users per week express suicidal thoughts on its platform.

A 2025 study evaluating some popular chatbots’ direct responses to suicide-related queries found inconsistencies in how those queries were addressed. Another study found several causes for concern related to consumer data protection, including a lack of transparency and accountability in how that data is used. The risks associated with using  generative AI for mental health purposes can be particularly acute for adolescents, because they may become dependent on the technology, and for older adults and individuals with existing mental illnesses due to their susceptibility to misinformation and exploitation.

Regulatory approaches and guidance to mental health providers

Consumers’ rapid adoption of chatbots and AI applications in healthcare delivery have been outpacing the legal and regulatory frameworks for these products. Currently, there are no uniform or mandatory national practice standards or guardrails around the scope, safety, or effectiveness regarding the use of these tools in the mental health field, and federal agencies have not issued comprehensive regulations for these products.

In the absence of federal regulations, some states have attempted to advance policies to govern how AI is applied in mental health, primarily focusing on clinician oversight, harm prevention, privacy protections, and user autonomy involving transparency, disclosure, and informed consent. For instance, Tennessee, Nevada, and Illinois have imposed restrictions or prohibitions on how AI systems are used or advertised to be used in mental healthcare delivery, including prohibiting AI systems from presenting as qualified mental health professionals and prohibiting providers from using AI for therapeutic decision-making. New York also passed a law requiring chatbot manufacturers to include protocols that, at minimum, detect user expressions of suicidal thoughts or self-harm and refer users in crisis to appropriate service providers, including suicide prevention and behavioral health crisis hotlines.

Professional associations such as the American Psychological Association and the American Counseling Association, academic institutions like the Stanford Institute for Human-Centered Artificial Intelligence and Harvard Medical School, and other public health stakeholders are also stepping in to provide recommendations, guidance, and training for the use of AI in mental healthcare to help ensure ethical, effective, and safe uses of these products.

As suicide reenters the top 10 causes of death in the United States and gaps in mental healthcare access remain, researchers, policymakers, patients, and the AI industry have the opportunity to collaborate and ensure that AI tools fulfill their potential for expanding access to mental healthcare while also protecting patient safety.  

Farzana Akkas works with The Pew Charitable Trusts’ suicide risk reduction project.