The Benefits of Utilizing a Surgical AI System

Saturday - May 13 2023, 08:11 UTC - 1 year ago

SAIS, a computer-based AI system developed by Caltech and USC, provides objective performance evaluation for surgeons, offering guidance for skill development, and the ability to explain assessments by pointing to video clips. Early tests of the system yielded unintended bias. The goal is to improve patient outcomes and make human surgeons better through AI.

The Surgical AI System (SAIS) developed by Caltech and Keck Medicine of USC urologists provides objective performance evaluations to surgeons, aiming to improve their skills and patient outcomes. By analyzing video footage, SAIS offers guidance on skill improvement and justifies its assessments with detailed feedback. Researchers are addressing unintended bias by focusing the AI on pertinent aspects of the surgical video.

When surgeons are trained, they usually need the supervision of more experienced doctors who can mentor them on their technique. That may be changing due to a new artificial intelligence system developed by Caltech researchers and Keck Medicine of USC urologists that aims to provide valuable feedback to surgeons on the quality of their work.

The goal of the new Surgical AI System (SAIS) is to provide surgeons with objective performance evaluations that can improve their work and, by extension, the outcomes of their patients. When provided with a video of a surgical procedure, SAIS can identify what type of surgery is being performed and the quality with which it was executed by a surgeon.

The system was introduced through a series of articles in the journals Nature Biomedical Engineering, npj Digital Medicine, and Communications Medicine, which were published concurrently at the end of March 2023.

"In high stakes environments such as robotic surgery, it is not realistic for AI to replace human surgeons in the short term," says Anima Anandkumar, Bren Professor of Computing and Mathematical Sciences and senior author of the studies. "Instead, we asked how AI can safely improve surgical outcomes for the patients, and hence, our focus on making human surgeons better and more effective through AI." .

SAIS was trained using a large volume of video data that was annotated by medical professionals. Surgeons’ performances were assessed down to the level of individual discrete motions, i.e., holding a needle, driving it through tissue, and withdrawing it from tissue. After training, SAIS was tasked with reviewing and evaluating surgeons’ performance during a wide range of procedures using video from a variety of hospitals.

"SAIS has the potential to provide surgeon feedback that is accurate, consistent, and scalable," says Dani Kiyasseh, lead author of the studies, a former postdoctoral researcher at Caltech and now a senior AI engineer at Vicarious Surgical. The hope, according to the researchers, is for SAIS to provide surgeons with guidance on what skill sets need to be improved.

To make the tool more valuable for surgeons, the team developed the AI’s ability to justify its skill assessments. The AI can now inform surgeons about their level of skill and provide detailed feedback on its rationale for making that assessment by pointing to specific video clips.

"We were able to show that such AI-based explanations often align with explanations that surgeons would have otherwise provided," Kiyasseh says. "Reliable AI-based explanations can pave the way for providing feedback when peer surgeons are not immediately available." .

Early on, researchers testing SAIS noted that an unintended bias crept into the system in which the AI sometimes rated surgeons as more or less skilled than their experience would otherwise indicate based solely on an analysis of the video.