Attention in surgical phase recognition for endoscopic pituitary surgery: Insights from real-world data

Abstract

[EN] Background and objective Surgical Phase Recognition systems are used to support the automated documentation of a procedure and to provide the surgical team with real-time feedback, potentially improving surgical outcome and reducing adverse events. The objective of this work is to develop a model for endoscopic pituitary surgery, a challenging procedure for phase recognition due to the high variability in the order of surgical phases. Methods A dataset of 69 pituitary endoscopic videos was collected and labelled by two surgeons in seven different phases. The architecture proposed comprises a Convolutional Neural Network to identify spatial features in individual frames, and a Segment Attentive Hierarchical Consistency Network (which combines Temporal Convolutional Networks with attention mechanisms) to learn temporal relationship information between frames and segments at different temporal scales. Finally, predictions are refined with an adaptative mode window. Results We have built and made publicly available the largest pituitary endoscopic surgery database to date, named PituPhase. We have built a model with a 73 % accuracy (75 % using a 10 s relaxed boundary). This result is comparable to other state-of-the-art methods in this surgical domain despite the challenges of the dataset (only 10 % of the videos are complete and only 3 % present all phases in the same order, versus 90 % and 50 % respectively in other studies). Conclusions Attention mechanisms in combination with Temporal Convolutional Networks and adaptive mode windows improve the performance of Surgical Phase Recognition systems and are robust to missing video sections and high variability in phase order.