Interpreting deep learning models for glioma survival classification using visualization and textual explanations.

Abstract

[EN] Background Saliency-based algorithms are able to explain the relationship between input image pixels and deeplearning model predictions. However, it may be difcult to assess the clinical value of the most important image features and the model predictions derived from the raw saliency map. This study proposes to enhance the interpret¿ ability of saliency-based deep learning model for survival classifcation of patients with gliomas, by extracting domain knowledge-based information from the raw saliency maps. Materials and methods Our study includes presurgical T1-weighted (pre- and post-contrast), T2-weighted and T2-FLAIR MRIs of 147 glioma patients from the BraTs 2020 challenge dataset aligned to the SRI 24 anatomical atlas. Each image exam includes a segmentation mask and the information of overall survival (OS) from time of diag¿ nosis (in days). This dataset was divided into training (n = 118) and validation (n = 29) datasets. The extent of surgi¿ cal resection for all patients was gross total resection. We categorized the data into 42 short (mean µ = 157 days), 30 medium (µ = 369 days), and 46 long (µ = 761 days) survivors. A 3D convolutional neural network (CNN) trained on brain tumour MRI volumes classifed all patients based on expected prognosis of either short-term, mediumterm, or long-term survival. We extend the popular 2D Gradient-weighted Class Activation Mapping (Grad-CAM), for the generation of saliency map, to 3D and combined it with the anatomical atlas, to extract brain regions, brain volume and probability map that reveal domain knowledge-based information. Results For each OS class, a larger tumor volume was associated with a shorter OS. There were 10, 7 and 27 tumor locations in brain regions that uniquely associate with the short-term, medium-term, and long-term survival, respec¿ tively. Tumors located in the transverse temporal gyrus, fusiform, and palladium are associated with short, medium and long-term survival, respectively. The visual and textual information displayed during OS prediction highlights tumor location and the contribution of diferent brain regions to the prediction of OS. This algorithm design feature assists the physician in analyzing and understanding diferent model prediction stages. Conclusions Domain knowledge-based information extracted from the saliency map can enhance the interpret¿ ability of deep learning models. Our fndings show that tumors overlapping eloquent brain regions are associated with short patient survival.