AI and extended reality successfully train surgical trainees without instructors

Mount Sinai researchers have demonstrated the effectiveness of teaching surgical trainees a difficult procedure using artificial intelligence (AI) algorithms and an extended-reality headset without the presence of an instructor. All of the 17 trainees in the study achieved surgical success.

The novel study, published in Journal of Medical Extended Reality, drew highly favorable reviews from student participants who tested the deep learning model. The results carry significant implications for future training of residents and surgeons, as well as for the even broader field of autonomous learning within medicine.

For the first time, we created an AI model linked to an extended-reality headset to prove that a critical step in a kidney cancer procedure could be done with 99.9 percent accuracy. We believe our study offers early proof that AI programs that substitute for proctors, who teach resident physicians, can reduce training costs and ultimately improve the quality, efficiency, and standardization of that instruction."

Nelson Stone, MD, Clinical Professor of Urology, Radiation Oncology, and Oncological Sciences at the Icahn School of Medicine at Mount Sinai, and corresponding author of the study

Surgical training of residents has traditionally required the presence of a teaching proctor alongside the student physician in the operating room, which can result in inconsistent skills acquisition. Dr. Stone and his team, which included researchers from the Department of Neurosurgery at the University of Rochester Medical Center in upstate New York, explored an alternative training system using AI programs they developed, including ESIST (educational system for instructionless surgical training). This model coupled deep learning methodology with a custom-designed extended-reality headset worn by the 17 participants to stream surgical instructions and video content before their eyes, while allowing their hands to remain free to practice the intricate procedure.

The operation simulated a partial nephrectomy procedure designed to remove a cancerous portion of a kidney, including placing a clamp on the renal artery. For this replication, researchers created a "phantom" kidney from 3D printed casts of an anonymized patient's computerized tomography (CT) scans. The casts were filled with water-based polymers and assembled to create a partial nephrectomy model with kidney tumors. While students practiced, the system's sophisticated first-person camera continuously monitored their training, providing real-time feedback and projecting corrective prompts as part of its skills assessment capability.

"Above all, our study proved that a complex procedure like a partial nephrectomy could be effectively taught to surgical trainees using a simulated model, without the presence of an instructor," noted Dr. Stone. "This finding addresses an urgent need resulting from the shortage of trainers and supervisors to educate physicians on new medical devices and techniques, and from the severe time constraints on attending physicians to train residents pursing surgical careers."

Another major advantage of advanced teaching technology, added Dr. Stone, is that it allows future surgeons to become proficient in procedures outside the operating room, thus helping to reduce the risk of surgical errors. "From the patient's point of view, we hope this study will provide reassurance that the technology can be leveraged to greatly improve surgical proficiency, while reducing surgical errors," said Dr. Stone.

The next step for Mount Sinai researchers is to use the AI algorithm technology they developed to build more complex synthetic cadaver models to train students in entire procedures, rather than just one component, as reported in the study. The team was encouraged by a survey it conducted after the training, which found that 100 percent of the participants believed the program had great educational value.

"Our investigation suggests that AI systems could indeed play an important complementary role in shaping the future of surgical education in this country," asserts Dr. Stone. "The public should be reassured that the pathway to autonomous learning we investigated in this small study could eventually lead to significant cost savings and improved patient outcomes and, importantly, to the cultivation of a highly skilled new generation of surgeons."

The study's authors, as listed in the journal, are Jonathan J. Stone, Nelson N. Stone, Steven H. Griffith, Kyle Zeller, and Michael P. Wilson.

All authors, except Kyle Zeller, hold equity in Viomerse.

The research was funded by the National Institute of Biomedical Imaging and Bioengineering (grant 1R41EB026358-01A1) and the National Science Foundation (grant 1913911).