The findings, published in npj Digital Medicine, suggest this technology could help surgeons refine their techniques and improve patient outcomes.

The AI system, called Frame-to-Outcome (F2O), analyzes video recorded during the nerve-sparing portion of robot-assisted prostate surgery, when surgeons work to preserve the nerves responsible for sexual function. Rather than relying on experts to manually evaluate each procedure, the system automatically identifies patterns in a surgeon’s movements—called “surgical gestures”—and uses them to predict patient recovery.

Prior studies of these surgical gestures demonstrated a strong relationship between the gestures performed by the surgeon—such as the sequence of instruments used or the speed of stretching nerves to move them aside—and the patient’s outcome. By analyzing the gestures used during surgery, the AI system made determinations about whether the patient will be more or less likely to have a good outcome. Until now, this type of analysis required labor-intensive review by trained human observers.

“Our goal isn’t simply to predict who will recover,” said Andrew Hung, MD, corresponding author of the study and professor of Urology at Cedars-Sinai. “We want to identify the surgical techniques with the best outcomes so surgeons can learn, refine and improve care for our future patients.”

Investigators found that F2O matched expert human reviewers in predicting patient outcomes while dramatically reducing the time required to analyze surgical performance. The technology could eventually provide surgeons with objective feedback on the techniques most closely associated with successful patient recovery.

To develop the system, researchers trained the AI system using videos annotated by human analysts from 294 surgeries from 23 surgeons across four international centers. Then they tested it on an additional 29 surgeries and found its predictions closely matched those of expert human reviewers.

“By identifying and interpreting the gestures that result in positive patient outcomes, we can offer surgeons insights that can help improve surgical performance and patient care,” Hung said.

The study was a collaboration between the Department of Urology, the Department of Computational Biomedicine and the Center for Artificial Intelligence Research and Education (CAIRE) at Cedars-Sinai.

“This work highlights what we can achieve when surgeons and computational biomedicine experts work together toward a shared clinical goal," said Jason Moore, PhD, chair of the Cedars-Sinai Department of Computational Biomedicine and director of CAIRE. "Bringing together these disciplines allowed the team to build something that is technically rigorous and genuinely meaningful for surgeons—and their patients."

Additional Cedars-Sinai authors include Xi Li, Nicholas Matsumoto, Jay Moran, Miguel E. Hernandez, Cherine Yang, Jeanine Kim, Jasmine Lin, Peter Wager, Ujjwal Pasupulety and Atharva Deo.

Other authors include Alvin C. Goh, Christian Wagner and Geoffrey A. Sonn.

Funding: Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA273031. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Cedars-Sinai Health Sciences University is advancing groundbreaking research and educating future leaders in medicine, biomedical sciences and allied health sciences. Learn more about the university.

Investigators coined the term surgical gestures to describe the smallest moment-to-moment movements made by a surgeon to complete a procedure.

“These surgical gestures are as minute as a single movement of an instrument, a single cut of tissue or a single retraction of tissue, for example,” said Andrew Hung, MD, a urologic surgeon, associate professor of Urology at Cedars-Sinai and senior author of the study. “The gestures work something like an alphabet. You can string them together to form a surgical procedure the way you string letters together to form words.”

Investigators reviewed surgical video of 157 patients undergoing robotic-assisted radical prostatectomy, then followed up to determine whether patients recovered their sexual function one year after the procedures.

“Radical prostatectomy is the most common treatment for localized prostate cancer and robotic-assisted prostatectomy is the most common surgical procedure performed in the U.S.,” said Hyung Kim, MD, chair of the Department of Urology at Cedars-Sinai. “These findings have the potential to change outcomes for tens of thousands of patients each year.”

During these procedures, where the prostate is removed but the nerves responsible for sexual function are preserved, the surgeon is faced with a task much like peeling an orange, Hung said.

“Imagine the nerves as the orange peel,” Hung said. “We are trying to separate the peel from the fruit without causing injury or harm and it is a delicate act. This is a procedure we’ve been doing for decades, yet 27% to 81% of patients do not recover their sexual function, so we have room to improve.”

Because it takes as much as a year of recovery to know whether a patient’s sexual function will return, it has been difficult for surgeons to connect the dots between actions during surgery and a patient’s outcome, Hung said. But robotic procedures, which are routinely recorded for training purposes, offer a unique opportunity to capture a surgeon’s-eye view. And investigators were able to link use of certain gestures—and avoidance of others—to recovery of sexual function in patients.

“One example is that during a procedure a surgeon might retract or stretch the nerves in order to move them aside, and we found that increased use of retraction was linked with patients’ failure to recover sexual function,” Hung said. “This could be because that pressure causes paralysis of the nerves even though they are physically preserved.”

The investigators’ goal is to build a library of evidence linking surgical gestures to patient outcomes, ultimately changing surgical practice and improving results.

“Technique is passed down from one generation of surgeons to the next, but without data to back up what mentors are teaching their students,” said Hung, who is also an associate professor of Computational Biomedicine. “Until now, we haven’t had the right tools to objectively assess surgical technique. This study is meant to connect the dots between what surgeons do and how well their patients recover.”

There are few surgical performance measures that accurately predict patient outcomes, according to Cristina Ferrone, MD, chair of the Jim and Eleanor Randall Department of Surgery at Cedars-Sinai and interim co-director for Cancer Clinical Services at Cedars-Sinai Cancer.

“These findings and future studies that build on this study could eventually change our approach to surgery and training surgeons,” Ferrone said. “By developing a clear understanding of core elements that translate to positive outcomes, it might be possible to improve results for patients across the surgical spectrum.”

Additional Cedars-Sinai authors include John R. Heard, MD; Umar Ghaffar, MD; and Cherine H. Yang, BA.

Other authors include Runzhuo Ma, MD; Melissa Assel, PhD; Christian Wagner, MD; Geoffrey A. Sonn, MD; Alvin C. Goh, MD; Shady Saikali, MD; Vipul Patel, MD; Andrew Vickers, PhD; and Jim C. Hu, MD.

Funding: This project was supported by a grant from the National Cancer Institute (R01CA273031).

Cedars-Sinai Health Sciences University is advancing groundbreaking research and educating future leaders in medicine, biomedical sciences and allied health sciences. Learn more about the university.