Automated Suturing Skill Assessment in Robot-assisted Surgery from Endoscopic Videos using Clinically-guided Evaluation Criteria

Surgery continues to be perceived as an art, where proficiency is primarily achieved through years of experience. Artificial Intelligence research has yielded insight into the performance of expert surgeons and their associations with patient outcomes. Clinician expertise has led to the development of systematic assessments for fundamental skills (e.g., End-to-End Assessment of Suturing Expertise [EASE]) that contribute to positive outcomes. However, evaluating these skills requires manual expert review of endoscopic videos and is prone to inconsistencies between human raters. In this work, we present AutoEASE, the first end-to-end pipeline to automatically assess suturing performance from raw endoscopic video data using EASE rubrics. Our system utilizes a Mixture of Expert models (MoE) ; Multiscale vision transformers and 3D convolutional neural networks trained on Robot-assisted Radical Prostatectomy videos with over 13000 data points. For a given stitch clip, the MoE pipeline first determines each phase (needle handling, driving, withdrawal) of a continuous stitch and predicts a binary score (fail / ideal) for seven sub-skills based on rubrics defined in EASE. AutoEASE achieves 0.98 AUC while detecting each phase. For EASE score prediction, the complete end-to-end pipeline attains $\geq$ 0.77 AUC in sub-skills associated with needle handling and driving. The promising performance of AutoEASE at the individual stitch level demonstrates the feasibility of developing more sophisticated assessment and reporting tools for complete surgical procedures objectively and at scale.