Current Issue Cover
面向多模态MRI脑胶质瘤区域三维分割与生存期预测的级联U-Net网络

余力1, 刘宵雪2, 闫朝阳1, 李建瑞2, 张志强2, 黄韫栀1, 徐军1(1.南京信息工程大学人工智能学院医学人工智能联合研究院, 南京 210044;2.东部战区总医院影像科, 南京 210002)

摘 要
目的 针对常见方法对脑胶质瘤的肿瘤分割和生存预测需要单独建模的问题,提出一种带有变分自编码器(variational auto-encoder,VAE)分支的两阶段级联U-Net算法,旨在分割肿瘤的同时提取鲁棒的特征预测患者生存期,有助于患者的精准治疗。方法 提出的两阶段级联U-Net网络,第1阶段实现初步粗分割,第2阶段实现精细化分割。此外,在第2阶段添加变分自编码器分支以提取更加鲁棒的特征并提高模型泛化性。其中,变分自编码器分支获取的特征被送入随机森林算法以进行生存期预测。另外,在两个阶段的解码器部分都添加了SE (squeeze-and-excitation)-残差模块以及注意力门模块,提高了分割精度。结果 在Brain Tumor Segmentation (BraTS)竞赛官网分别评估了本文方法在BraTS2020验证集上分割以及总体生存期预测两大任务的结果,本文算法在该验证集的全肿瘤区域、肿瘤核心区域以及增强型肿瘤区域分别取得了90.66%、85.09%和79.02%的Dice相似系数。相较3DU-Net在3个肿瘤子区域的Dice相似系数分别提高了4.3%、1.37%和5.36%。对于总体生存期的预测准确率达到了55.2%,相较性能最佳算法的预测准确率虽然低3.4%,但是从均方误差这一评估指标来看略有优势。结论 为了防止出现过拟合,提出带有VAE分支的两阶段级联U-Net网络结构,同时该分支能够提取肿瘤的鲁棒特征用于后续生存分析。实验表明,该网络能够较为准确地分割脑肿瘤,并且其提取的特征能够准确地预测患者生存期,可以为临床医生进行脑胶质瘤诊断提供有力的参考。
关键词
Brain glioma tumor segmentation and survival prediction from multi-modality MRIs via cascaded U-Net

Yu Li1, Liu Xiaoxue2, Yan Chaoyang1, Li Jianrui2, Zhang Zhiqiang2, Huang Yunzhi1, Xu Jun1(1.Institute for AI Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China;2.Department of Radiology, General Hospital of Eastern Theater Command, Nanjing 210002, China)

Abstract
Objective Glioma is the most common primary brain tumor with the highest mortality. The survival rate of different degrees of gliomas varies greatly. The magnetic resonance imaging based (MRI-based) scan imaging in the context of tumor substructure is composed of diversified intensity distributions, reflecting different tumor biological characteristics. Diagnosis grading and survival time prediction are conducted in terms of accurate detection and segmentation of abnormal tumor tissues. Radiomics aims to mine medical image data via a wide range of quantitative features extraction nowadays. The radiomics-based analyses are becoming essential for the medical prognosis. There is some constraints in designing and extracting features for traditional image omics. Deep learning methods are subjected of manual designing and extracting features. Deep learning convolutional neural networks (CNNs) has its priority in U-Net based image segmentation currently. Due to the time-consuming and labor-intensive labeling of gliomas and the lack of a large amount of supervision information, a U-Net based variational-auto-encoder (VAE) branch and the self-supervision information can deal with the over-fitting of the model effectively. The VAE branch can extract the potential robust features of images to resolve traditional image omics. A two-stage cascaded U-Net algorithm is demonstrated in context of VAE branch, which can realize the automatic segmentation of gliomas for diagnosis grading and the extraction of robust features derived from subsequent survival time prediction. Method In the first stage, the network predicts the coarse segmentation results in terms of the input of multi-modal MR images. The initial results and the multi-modal MR images are integrated as the input of the second stage for the network to improve the segmentation results. Moreover, a VAE branch is melted in the second stage of the network to obtain robust representative features and improve the generalization of the model. The representative features are used to predict the human life span based on the random forest algorithm. The traditional convolution gives the same weights to all channels of the input feature maps by default for feature fusion, while the squeeze-and-excitation (SE) module can learn the importance of different channels, so the SE module is evolved to the basic unit of this demonstrated algorithm. Furthermore, the prediction illustrates large differences in shape and size between patients, the cascaded network that detects and then segment can lead to the redundancy of model parameters. The proposed network adds an attention gate mechanism to the additive jump connection of the traditional U-Net, which achieves the same effect as the method that detects and then segment without increasing the amount of parameters. In addition, when the enhanced tumor area is small, this algorithm analysis is probably mislead the necrotic area and the non-enhanced tumor area as the enhanced tumor area. When the predicted enhanced tumor area is less than 500 voxels, the enhanced tumor area is substituted with necrotic and non-enhancing tumor area. This research method illustrates a random forest-based regressor for the prediction of the overall survival time and a random forest-based classifier for the prediction of category of the overall survival time at the end. The premise requires the use of low-dimensional potential features extracted by VAE and the clinical features of the patient's age, and feature option is implemented via the recursive feature elimination method based on random forest regression. When the prediction result of the regression model is inconsistent with the prediction category of the classification model, the final prediction result is divided into a fixed overall survival time according to the prediction category if the probability predicting the category exceeds 80% The short-term corresponds to 10 months, the mid-term corresponds to 12.5 months, and the long-term corresponds to 15 months. Result Our research analysis implements a five-fold cross validation on BraTS2020 training set. The generated result is the average of 5 validation results. The method achieves Dice similarity coefficients of 90.66%, 85.09%, and 79.02% in the whole tumor area, tumor core area and enhanced tumor area in the validation set, respectively. In terms of the clinical feature of patient's age, the overall survival prediction accuracy rate reaches 41.4%, and the potential extracted features in the VAE branch, the accuracy rate could reach 55.2%. Although the accuracy rate is 3.5% lower than that of the first-ranked method, this proposed method has a slight advantage in terms of the mean square error. Conclusion the research proposes a two-stage cascaded U-Net network with a VAE branch to resolve over-fitting issue. Simultaneously, the branch can also extract the potential features of the tumor to predict the survival time. The test results demonstrate that the network can segment brain tumors more accurately, and the potential extracted features can predict suitable patient survival time, which can provide a powerful reference for clinicians in the diagnosis of gliomas.
Keywords

订阅号|日报