Investigators studied four large language models (LLMs), a category of AI algorithms trained on enormous amounts of data, which enables them to understand and generate human language. In medicine, LLMs are drawing interest for their ability to quickly evaluate and recommend diagnoses and treatments for individual patients.

The study found that the LLMs, when presented with hypothetical clinical cases, often proposed different treatments for psychiatric patients when African American identity was stated or simply implied than for patients for whom race was not indicated. Diagnoses, by comparison, were relatively consistent. The findings were published in the peer-reviewed journal NPJ Digital Medicine.

“Most of the LLMs exhibited some form of bias when dealing with African American patients, at times making dramatically different recommendations for the same psychiatric illness and otherwise identical patient,” said Elias Aboujaoude, MD, MA, director of the Program in Internet, Health and Society in the Department of Biomedical Sciences at Cedars-Sinai and corresponding author of the study. “This bias was most evident in cases of schizophrenia and anxiety.”

Among the disparities the study uncovered were the following:

• Two LLMs omitted medication recommendations for an attention-deficit/hyperactivity disorder case when race was explicitly stated, but they suggested them when those characteristics were missing from the case.
• Another LLM suggested guardianship for depression cases with explicit racial characteristics.
• One LLM showed increased focus on reducing alcohol use in anxiety cases only for patients explicitly identified as African American or who had a common African American name. 

Aboujaoude suggested the LLMs showed racial bias because they reflected bias found in the extensive content used to train them. Future research, he said, should focus on strategies to detect and quantify bias in artificial intelligence platforms and training data, create LLM architecture that resists demographic bias, and establish standardized protocols for clinical bias testing.

“The findings of this important study serve as a call to action for stakeholders across the healthcare ecosystem to ensure that LLM technologies enhance health equity rather than reproduce or worsen existing inequities,” said David Underhill, PhD, chair of the Department of Biomedical Sciences at Cedars-Sinai and the Janis and William Wetsman Family Chair in Inflammatory Bowel Disease. “Until that goal is reached, such systems should be deployed with caution and consideration for how even subtle racial characteristics may affect their judgment.”

Other Cedars-Sinai authors include: Ayoub Bouguettaya, PhD. Other authors include Elizabeth M. Stuart, PhD.

Cedars-Sinai Health Sciences University is advancing groundbreaking research and educating future leaders in medicine, biomedical sciences and allied health sciences. Learn more about the university.

The multi-institutional study was published in the peer-reviewed journal JAMA Internal Medicine.

The addiction consultation service, called the Substance Use Treatment and Recovery Team (START), pairs experts in addiction treatment: a physician with a social worker or case manager. The team provides each patient with diagnostic assessments, recommendations for treatments, psychosocial support, connections to post-hospital medical care and follow-up telephone calls for one month.

“Our program addresses a major challenge across hospitals,” said Itai Danovitch, MD, chair of the Department of Psychiatry and Behavioral Neurosciences at Cedars-Sinai and senior author of the study. “Even though effective medications exist for opioid use disorders, only a small percentage of hospitalized patients begin treatment during their stay or connect with services after discharge.”

Danovitch, START lead principal investigator, said the service offers a supportive, patient-centered intervention that reduces stigma, promotes engagement and increases access to both treatment and follow-up services.

The co-led study was done in collaboration with RAND and conducted from 2021 through 2023 at three geographically distinct U.S. hospitals: Cedars-Sinai in Los Angeles; University of New Mexico Hospital in Albuquerque, New Mexico; and Baystate Medical Center in Springfield, Massachusetts.

The study included 325 hospitalized patients 18 or older who had a probable diagnosis of opioid use disorder. Of the patients enrolled,164 were randomly assigned to the START program while 161 received the usual hospital care.

The data shows that 57.3% of the START patients began medication treatment for opioid use disorder while in the hospital compared with 26.7% of the patients in the usual care group. Additionally, the study showed that 72% of the START patients accessed follow-up care within 30 days after discharge from the hospital compared with 48.1% of the control group.

Prior studies have shown that addiction consultation services are associated with improvements in care and outcomes, but this new study was the first multisite, randomized, controlled trial to compare the effects of such services to usual care, Danovitch said.

Opioid use disorder is a persistent problem. Nearly 80,000 people died from opioid overdoses in the U.S. in 2023, according to the U.S. Centers for Disease Control and Prevention. Between 2002 and 2018, hospital admissions for opioid use disorder rose significantly—from nearly 302,000 to nearly 942,000.

“Hospitalization provides a crucial window to involve patients in addiction treatment when they might be most open to it, particularly after experiencing health-related consequences of their substance use,” said Jeffrey A. Golden, MD,  director of the Burns and Allen Research Institute and executive vice dean for Research and Education at Cedars-Sinai. “These important findings show how the medical community can significantly boost this engagement and help find solutions for the national opioid epidemic.”

Other Cedars-Sinai authors include: Mia W. Mazer, BSc; Teryl Nuckols, MD, MSHS; and Waguih William IsHak, MD.

Other authors include: Allison J. Ober, PhD; Cristina Murray-Krezan, PhD; Kimberly Page, PhD; Peter D. Friedmann, MD, MPH; Jess Anderson, MS; Karen Chan Osilla, PhD; Stephen Ryzewicz, MD; Sergio Huerta, MD: Randall A. Hoskinson, MA; Richard Garvey, MA; Alexandra Peltz, BA; Katherine E. Watkins, MD, MSHS; and Louis T. Mariano, PhD.

Funding: Support for the study was provided by the National Center for Advancing Translational Sciences and National Institute on Drug Abuse, both parts of the National Institutes of Health, under grant award number U01TR002756-01A1. This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Cedars-Sinai Health Sciences University is advancing groundbreaking research and educating future leaders in medicine, biomedical sciences and allied health sciences. Learn more about the university.

Both studies used an application, developed at Cedars-Sinai, that combines AI and virtual reality (VR) goggles. It features avatars that are “trained” by AI to conduct talk therapy with patients in virtual and relaxing environments.

In the first study, published in the Journal of Medical Extended Reality, investigators used the VR application to deliver mental health therapy to 20 patients with alcohol-associated cirrhosis, a serious liver disease that can result from long-term excessive alcohol consumption.

Each patient received a 30-minute counseling session from a virtual therapist avatar that had received AI training in motivational interviewing, cognitive behavioral therapy and other techniques to help patients alter their behaviors. More than 85% of the patients said they found the sessions beneficial and 90% expressed interest in using virtual therapists again.

“For individuals awaiting liver transplants for cirrhosis, alcohol addiction remains a high-risk factor,” said Brennan Spiegel, MD, MSHS, professor of Medicine, director of Health Services Research at Cedars-Sinai and the corresponding author for both studies. “We see VR as a way to augment traditional interventions, which often fall short due to a shortage of mental health professionals, societal stigmatizing of alcoholism and other factors.” Spiegel also participated in recent research demonstrating that VR experiences can even modulate stress levels and immune responses in participants.

In the second study, published in the journal Cyberpsychology, Behavior, and Social Networking, investigators presented virtual therapists with virtual patients that had been trained by AI to emulate people seeking professional help for anxiety or depression. In each simulated conversation, the virtual therapist was randomly informed of a different sociodemographic profile of the patient based on age, gender, race, ethnicity and annual income. A control group without assigned identities was also included.

The investigators used a standard scale known as “tone analytics” to rate the tone or mood of the language used by the virtual therapist. In repeated samplings involving more than 400 conversations, no significant difference was found in the therapist’s tone score based on a virtual patient’s profile or the absence of a profile. “This data suggests that with thoughtful design, AI can offer equitable and personalized care,” Spiegel said.

These studies highlight the leading role Cedars-Sinai is taking to adopt new technologies and to develop patient-centered approaches to health challenges, said  Peter Chen, MD, professor of Medicine, the Medallion Chair in Molecular Medicine and interim chair of the Cedars-Sinai Department of Medicine.

“These two studies underscore Cedars-Sinai’s commitment to exploring the tremendous potential of artificial intelligence for mental health therapy while ensuring that this technology does not perpetuate human biases in delivering healthcare,” Chen said.  “Cedars-Sinai has become a world leader in tackling this formidable challenge.”

Other Cedars-Sinai authors of the Journal of Medical Extended Reality study include Yee Hui Yeo, Allistair Clark, Muskaan Mehra, Itai Danovitch, Ju Dong Yang, Alexander Kuo, Hyun-Seok Kim, Aarshi Vipani, Yun Wang, Walid Ayoub, Hirsh Trivedi, Jamil S. Samaan and Omer Liran. Additional authors were Karen Osilla, Tiffany Wu, and Vijay H. Shah.

Disclosures: Spiegel and Liran are faculty members at Cedars-Sinai Medical Center and co-founders of VRx Health, Inc. Spiegel has no role in the company, has received no payments, royalties, or proceeds from the company, has fully divested all equity and interests in VRx Health, Inc., and has divested from any interest and any potential royalties. Liran is on the Board of VRx Health and maintains equity; he is also eligible to receive future royalties. The AI algorithm in this study was exclusively licensed by Cedars-Sinai to VRx Health for commercialization. Cedars-Sinai has the right to receive future royalty payments from VRx Health under this license agreement and owns stock in VRx Health. All other authors do not have conflict of interest.

Other Cedars-Sinai authors of the Cyberpsychology, Behavior, and Social Networking study include Yee Hui Yeo, Muskaan Mehra, Jamil S. Samaan, Joshua Hakimian, Allistair Clark, Karisma Suchak, Zoe Krut, Taiga Andersson and Omer Liran. Additional authors were Yuxin Peng and Susan Persky.

Disclosures: Spiegel and Liran are faculty members at Cedars-Sinai Medical Center and co-founders of VRx Health, Inc. The AI algorithm in this study was exclusively licensed by Cedars-Sinai to VRx Health for commercialization. Cedars-Sinai has the right to receive future royalty payments from VRx Health under this license agreement, and owns stock in VRx Health. All other authors do not have conflict of interest.

Follow Cedars-Sinai Academic Medicine on LinkedIn for more on the latest basic science and clinical research from Cedars-Sinai.

Heart failure affects nearly 6 million adults in the United States, and approximately 50% of heart failure patients experience symptoms of depression along with their condition. Past studies show patients with heart failure and depression have lower cardiac function, more emergency department visits and hospital admissions, higher caregiver burden, and poorer quality of life compared with patients with heart failure who are not depressed.

“The most important finding here is that patients experiencing depression have a choice in terms of their treatment between therapy or medications,” said Waguih W. IsHak, MD, vice chair of Education and Research in the Department of Psychiatry and Behavioral Neurosciences at Cedars-Sinai and first author of the study. “Patients who prefer not to be on medication can do behavioral activation therapy with similar results.”

The study, which was published in the peer-reviewed journal JAMA Network Open, followed more than 400 patients over the course of a year. Half of the study participants received antidepressant medication management to treat depression symptoms while the other half participated in behavioral activation psychotherapy, an evidence-based treatment for depression that promotes engagement in activities a patient finds enjoyable. 

The research was done in close collaboration with the Smidt Heart Institute at Cedars-Sinai, and its heart failure program led by Michele Hamilton, MD, vice chair of the Department of Cardiology in the Smidt Heart Institute and co-author of the study.

Investigators say there was no statistically significant difference between the effectiveness of the two methods, with each patient group experiencing a more than 50% reduction in the severity of depressive symptoms.

Study participants randomized to receive behavioral activation therapy worked with a therapist to develop a personalized list of activities that bring them joy and fulfillment, including having lunch with a friend, taking a walk, volunteering, listening to their favorite music—among other options—on a regularly scheduled basis. The treatment focused on increasing the level of enjoyable and rewarding activities the patient was doing every week for 12 weeks, and then incorporating them into their regular activity schedule.

Patients who received psychotherapy intervention showed a slight improvement in their physical and mental health-related quality of life, a secondary outcome monitored by the study. They also had fewer emergency department visits and spent fewer days hospitalized during the course of the study compared with the patients who were randomized to receive antidepressant medications.

“This study, which was conducted in a real-world setting, demonstrates that it is entirely feasible to incorporate psychiatric treatment into specialty medical care. Integrating psychiatric treatment into medical care is an effective way to reduce stigma, increase access and improve outcomes for people who struggle with mental health problems alongside their chronic medical conditions,” said Itai Danovitch, MD, MBA, chair of the Department of Psychiatry and Behavioral Neurosciences and a study co-author. “Most people who suffer from depression do not receive effective treatment. It is critical that we increase screening for psychiatric conditions and ensure that patients have access to high-quality and effective mental health care.”

Other Cedars-Sinai authors involved in the study are Michele A. Hamilton, MD; Samuel Korouri, BA; Marcio A. Diniz, PhD; James Mirocha, MS; Rebecca Hedrick, MD; Robert Chernoff, PhD; Jeanne T. Black, PhD; Harriet Aronow, PhD; Brigitte Vanle, MD, PhD; Jonathan Dang, MD; Gabriel Edwards, MD; Tarneem Darwish, MD; Gabrielle Messineo, LCSW; Stacy Collier, RN; Mia Pasini, RN; John G. Harold, MD; and Brennan Spiegel, MD, MSHS. Additional authors include Kaleab K. Tessema, MD, PhD; Michael K. Ong, MD; and Kenneth Wells, MD.

Funding: The study by was funded by the Patient-Centered Outcome Research Institute (PCORI) (award No.: 2017C2-7716; paid to Cedars-Sinai Medical Center).

Read more from the Cedars-Sinai Blog: Living With Heart Failure

The study, published in the Journal of Pain, compared patient education and clinical decision support for managing opioids. A simple patient education intervention improved overall satisfaction with patient communications, while a carefully crafted electronic alert showed greater potential to reduce high-dose opioid prescribing. No difference was found in pain interference between the two groups.

“The goal of our study was to help patients and doctors make safer decisions when prescribing opioids for chronic pain while also seeking to avoid the stigma sometimes attributed to using pain medications,” said Brennan Spiegel, MD, MSHS director of Health Services Research, George and Dorothy Gourrich Chair in Digital Health Ethics, and the study’s lead author. “We want to make sure we’re maximizing the appropriateness of dosing so we’re helping people manage their pain without doing harm.”

The study was funded by a Patient-Centered Outcomes Research Institute grant. One in 5 American adults experience chronic pain. Overdose deaths In the U.S. related to prescription opioids increased by over 4 times compared to 20 years, according to the National Institutes of Health, constituting a public health crisis.

“Patients and their physicians both struggle to manage chronic pain, particularly when opioids become involved,” said Itai Danovitch, MD, chair of the Department of Psychiatry and Behavioral Neurosciences. “We now understand that opioids are double-edged swords—when it comes to chronic pain, their harms are often greater than their benefits. Our study showed that both clinical decision support and patient education can have positive effects. We need continued support to help physicians adopt best practices, and we need effective education to empower patients to promote their own health.” 

Investigators compared the effectiveness of physician-directed clinical decision support administered through the electronic health record versus patient-directed education to promote the appropriate use of opioids through a cluster-randomized trial. 

The trial included 82 primary care physicians at Cedars-Sinai and 951 of their patients with chronic pain. They measured outcomes based on satisfaction with patient-physician communication, patient interference, physical function, depression, high-risk opioid prescribing, and co-prescribing of opioids and benzodiazepines.

The outcomes were the same in both groups on most measures, but patient education was linked to higher satisfaction with communications and the electronic alerts with lower opioid dosing.

Dodging Alert Fatigue

Cedars-Sinai has been on the forefront of using clinical decision support, Spiegel said. Too many alerts, however, can lead to them being ignored.

The alert in this study was a point-of-care reminder that would pop up on the computer screen to help identify if opioids were being prescribed but might not be the most appropriate intervention. 

“We worked very closely with our colleagues in primary care in the Medical Delivery Network to make sure the alert was appropriate,” Spiegel said. “We wanted them to have a direct role in crafting it and vetting it, which isn’t always the case with the alerts doctors routinely experience in the electronic health record.” 

In addition, to avoid alert fatigue, the alerts had clear action items rather than simply providing additional information. For example, the medical network runs and opioid withdrawal program, and the alerts had a button to refer patients to the program if appropriate.

As many as 85% of alerts are simply dismissed by clinicians, said Teryl Nuckols, MD, MSHS, director of the Division of Internal Medicine.

“That these alerts can be effective in reducing the rate of high dose opioids is a powerful finding,” she said. “It’s not easy to change clinician behavior, and the engagement work that was done seems to have made a significant difference.”

Scalable Interventions

The other arm of the study provided ready-made materials to patients, arming them with questions to ask should opioids be prescribed to them and informing them of the risks.

The study team also included a chronic pain patient who created a newsletter that was sent out to patient participants each month, providing updates about the research and information about the team. In addition, the research team formed a patient advisory panel who met quarterly to advise on all aspects of the study.

Ultimately, the study concluded that patient educational materials is an inexpensive, easily scaled intervention that can improve satisfaction with patient-provider communications. Clinical decision support is a more resource-intensive undertaking, but provided benefits that education alone could not. Using the combination in primary care warrants further study, Spiegel said.

Follow Cedars-Sinai Academic Medicine on Twitter for more on the latest basic science and clinical research from Cedars-Sinai.