Introduction:

Laparoscopic and robotic surgeons rely heavily upon surgical video for self-assessment and performance evaluation. Review of intraoperative video enables surgeons to study decision-making during key moments, identify pitfalls, develop best practices, and cultivate situational awareness. However, large-scale and efficient review of surgical video is resource-intensive, thereby limiting its use. Although artificial intelligence (AI) has shown promise in facilitating large-scale interpretation of medical imaging, current applications of AI to surgical video remain limited. We aim to develop a computer vision algorithm for automated identification of key surgical steps during minimally invasive partial nephrectomy (PN) and radical nephrectomy (RN).

Methods:

Retrospective surgical videos from laparoscopic and robotic PN and RN were reviewed by a team of medical image annotators under the supervision of a fellowship-trained urologic oncologist. Key surgical steps were manually annotated for each surgery, including preparation and port placement, adhesiolysis, initial dissection and bowel mobilization, tumor resection (for PN), renorrhaphy (for PN), and specimen retrieval/final inspection. A training dataset of manually annotated videos was then utilized to train an AI-enabled computer vision algorithm for automated detection of each of these key surgical steps. Accuracy of the AI algorithm (without human input) was assessed by comparing to human-annotated videos as the gold standard. AI model performance was measured on an overall as well as a per-step basis.

Results:

A total of 165 full-length surgical videos, including 63 PN and 102 RN, were included. The dataset was divided into training (n=98), internal validation (n=25), and test (n=42) cohorts. The overall AI model achieved 91.6% accuracy as compared to human video annotation. Individual per-step AI performance accuracy was: preparation and port placement 91.2%, adhesiolysis 40.8%, initial dissection and bowel mobilization 95.6%, tumor resection (for PN) 72.0%, renorrhaphy (for PN) 83.8%, and specimen retrieval/final inspection 91.2%.

Conclusion:

To our knowledge, this represents the first AI-enabled computer vision algorithm capable of autonomously identifying key surgical steps during minimally invasive kidney surgery based on surgical video alone. This may facilitate future efforts to utilize annotated surgical video for resident education, surgical quality assessment, and correlating intraoperative surgical events with post-operative outcomes.

Funding: N/A