The late-breaking study presented at the American College of Physicians Internal Medicine Meeting and published simultaneously in the Annals of Internal Medicine compared initial AI treatment recommendations to final recommendations of physicians who had access to the AI recommendations but may or may not have reviewed them.

“We found that initial AI recommendations for common complaints in an urgent care setting were rated higher than final physician recommendations,” said Joshua Pevnick, MD, MSHS, co-director of the Cedars-Sinai Division of Informatics, associate professor of Medicine and co-senior author of the study. “Artificial intelligence, as an example, was especially successful in flagging urinary tract infections potentially caused by antibiotic-resistant bacteria and suggesting a culture be ordered before prescribing medications.”

However, Pevnick said that while AI was shown to be better at identifying critical red flags, “physicians were better at eliciting a more complete history from patients and adapting their recommendations accordingly.”

The retrospective study was conducted using data from Cedars-Sinai Connect, a virtual primary and urgent care program that began in 2023. An extension of Cedars-Sinai’s in-person care, Cedars-Sinai Connect aims to expand virtual healthcare for patients in California through a mobile app that allows individuals to quickly and easily access Cedars-Sinai experts for acute, chronic and preventive care.

The study reviewed 461 physician-managed visits with AI recommendations from June 12 through July 14, 2024. Key medical issues addressed during these virtual urgent care visits involved adults with respiratory, urinary, vaginal, vision or dental symptoms.

Patients using the mobile app initiate visits by entering their medical concerns and, for first-time users, providing demographic information. An expert AI model conducts a structured dynamic interview, gathering symptom information and medical history. On average, patients answer 25 questions in five minutes.

An algorithm uses the patient’s answers as well as data from the patient's electronic health record to provide initial information about conditions with related symptoms. After presenting patients with possible diagnoses to explain their symptoms, the mobile app allows patients to initiate a video visit with a physician.

The algorithm also suggests diagnosis and treatment recommendations that can be viewed by the Cedars-Sinai Connect treating physician, though during the time of the study, Cedars-Sinai Connect required physicians to scroll down to view them.

“The major uncertainty of this study is whether physicians scrolled down to view the prescribing, ordering, referral or other management suggestions made by AI, and whether they incorporated these recommendations into their clinical decision-making,” said Caroline Goldzweig, MD, Cedars-Sinai Medical Network chief medical officer and co-senior author of the study. “The fact that the AI recommendations were often rated as higher quality than physician decisions, however, suggests that AI decision support, when implemented effectively at the point of care, has the potential to improve clinical decision-making for common and acute conditions.”

The AI system used for Cedars-Sinai Connect is developed by K Health, which created the technology to reduce the burdens of clinical intake and data entry, allowing doctors to focus more on patient care. K Health and Cedars-Sinai developed Cedars-Sinai Connect through a joint venture and collaborated on the research study. Investigators from Tel Aviv University, including first author Dan Zeltzer, PhD, also participated in the study.

“We put AI to the test in real-world conditions, not contrived scenarios,” said Ran Shaul, co-founder and chief product officer of K Health. “In the reality of everyday primary care, there are so many variables and factors—you’re dealing with complex human beings, and any given AI has to deal with incomplete data and a very diverse set of patients.”

Shaul said the investigators learned that if you train the AI on the treasure trove of de-identified clinical notes and use day-to-day provider care as an always-on reinforcement learning mechanism, “you can reach the level of accuracy you would expect from a human doctor.”

Other authors involved in the study include Dan Zeltzer, PhD; Zehavi Kugler, MD; Lior Hayat, MD; Tamar Brufman, MD; Ran Ilan Ber, PhD; Keren Leibovich, PhD; Tom Beer, MSc; and Ilan Frank, MSc.

This work was supported with funding by K Health.

Cedars-Sinai Health Sciences University is advancing groundbreaking research and educating future leaders in medicine, biomedical sciences and allied health sciences. Learn more about the university.

Through the new virtual healthcare option Cedars-Sinai Connect, healthcare professionals are available 24/7 for urgent issues, and same-day appointments are accessible for primary care—reducing wait times for those seeking medical attention.

“Cedars-Sinai Connect is a natural extension of the high-quality care Cedars-Sinai already provides to patients in person,” said Jill Martin, executive vice president of Cedars-Sinai Medical Network. “What could be more convenient than getting the care you need when you need it—in the palm of your hand—in your home or office?”

The new mobile app harnesses AI technology developed by K Health that empowers physicians by reducing the burdens of clinical intake and data entry, allowing doctors to focus more on patient care. K Health helps patients manage their acute, chronic and preventive care needs through a platform that harnesses medical knowledge and real patient experiences. K Health and Cedars-Sinai developed Cedars-Sinai Connect through a new joint venture.

The partnership gives new and existing patients expanded treatment options for a broad range of medical conditions from a dedicated team of virtual care providers. The app’s chat feature gathers health details and symptoms from patients to help assess their medical concerns and  presents that information to a clinician for review. This streamlines the intake process so providers can focus more on patient care. After the guided intake, patients can choose to connect with a clinician for an appointment via video call.

Cedars-Sinai Connect will accept all forms of insurance that Cedars-Sinai takes, including employer-provided plans.

“Patient care plans and visit information are integrated into the Cedars-Sinai health record, making it easy for patients to move between online and in-person care in a coordinated way, and for patients to access our broader network of specialists when needed,” said Caroline Goldzweig, MD, chief medical officer of Cedars-Sinai Medical Network. “This new platform will help us expand and enhance care for patients across California.”

K Health’s founders said the partnership with Cedars-Sinai will give patients greater access to world-class medical expertise, empowering people to take charge of their health by choosing when and where they get care.

“We’re excited to work with Cedars-Sinai, an esteemed institution that shares our vision to provide people with unfettered access to exceptional medical care around the clock,” said Ran Shaul, the company’s co-founder and chief product officer. “By fusing K Health’s technology with Cedars-Sinai’s renowned care, we are advancing primary care delivery by extending essential services to a wider spectrum of people in need while alleviating physician burnout.”

Read more on the Cedars-Sinai Blog: Virtual Second Opinion | Here’s What to Know