多尺度融合注意力机制的胆囊癌显微高光谱图像分类
高红民1,2, 朱敏1,2, 曹雪莹1,2, 李臣明1, 刘芹3, 许佩佩4,2(1.河海大学信息学部计算机与信息学院, 南京 211100;2.华东师范大学上海市多维度信息处理重点实验室, 上海 200241;3.南京大学医学院附属鼓楼医院肿瘤科, 南京 211108;4.南京大学医学院附属鼓楼医院血液科, 南京 211108) 摘 要
目的 胆囊癌作为胆道系统中一种恶性程度极高的肿瘤,早期诊断困难、预后极差,因此准确鉴别胆囊病变对早期发现胆囊癌具有重要意义。目前胆囊癌的诊断主要依赖于超声、CT(computed tomography)等传统影像学方法,但准确性较低。显微高光谱能够在获取生物组织图像信息的同时从生化角度对生物组织进行分析,从而实现对胆囊癌的早期诊断,相比于传统医学图像更具优势。因此,本文基于胆囊癌显微高光谱图像设计了一种基于多尺度融合注意力机制的网络模型,以提高分类准确率。方法 提出多尺度融合注意力模块(multiscale squeeze-andexcitation-residual,MSE-Res)。MSE-Res模块引入改进的多尺度特征提取模块实现通道维上特征的融合,用一个最大池化层和一个上采样层代替1 × 1的卷积层来提取图像的显著特征。为了弥补池化层丢失的局部信息,在跳跃连接中加入一个1 × 1的卷积层。在多尺度特征提取模块后,引入注意力机制来学习不同通道间特征的相关性,实现通道间特征的融合,并通过残差连接使网络在提取图像深层特征的同时避免出现过拟合现象。结果 在胆囊癌高光谱数据集上进行实验,本文模型的总体分类精度、平均分类精度和Kappa系数分别为99.599%、99.546%和0.990,性能优于SE-ResNet(squeeze-and-excitation-residual network)和Inception-SE-ResNet(inception-squeeze-andexcitation-residual network)。结论 本文提出的MSE-ResNet能够有效利用高光谱图像的空间信息和光谱信息,提高胆囊癌分类准确率,在对胆囊癌的医学诊断方面具有一定的研究价值和现实意义。
关键词
A micro-hyperspectral image classification method of gallbladder cancer based on multi-scale fusion attention mechanism
Gao Hongmin1,2, Zhu Min1,2, Cao Xueying1,2, Li Chenming1, Liu Qin3, Xu Peipei4,2(1.College of Computer and Information Engineering, Department of Information, Hohai University, Nanjing 211100, China;2.Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai 200241, China;3.Department of Oncology, Drum Tower Hospital, School of Medicine, Nanjing University, Nanjing 211108, China;4.Department of Hematology, Drum Tower Hospital, School of Medicine, Nanjing University, Nanjing 211108, China) Abstract
Objective Gallbladder carcinoma is recognized as one of the most malignant tumors in relevant to biliary system. Its prognosis is extremely poor,and only 6 months of overall average. It is challenged for missed diagnose because of the lack of typical clinical manifestations in early stage of gallbladder cancer. To clarify gallbladder lesions for early detection of gallbladder carcinoma accurately,current gallbladder cancer-related diagnosis is mainly focused on the interpretation of digital pathological section images(such as b-ultrasound,computed tomography(CT),magnetic resonance imaging (MRI),etc.)in terms of the computer-aided diagnosis(CAD). However,the accuracy is quite lower because the molecular level information of diseased organs cannot be obtained. Micro-hyperspectral technology can be incorporated the features of spectral analysis and optical imaging,and it can obtain the chemical composition and physical features for biological tissue samples at the same time. The changes of physical attributes of cancerous tissue may not be clear in the early stage,but the changes of chemical factors like its composition,structure and content can be reflected by spectral information. Therefore,micro hyperspectral imaging has its potentials to achieve the early diagnosis of cancer more accurately. Micro-hyperspectral technology,as a special optical diagnosis technology,can provide an effective auxiliary diagnosis method for clinical research. However,it can provide richer spectral information but large amount of data and information redundancy are increased. To develop an improved accuracy detection method and use the rich spatial and hyperspectral information effectively,we design a multi-scale fusion attention mechanism-relevant network model for gallbladder canceroriented classification accuracy optimization. Method The multiscale squeeze-and-excitation-residual(MSE-Res)can be used to realize the fusion of multiscale features between channel dimensions. First,an improved multi-scale feature extraction module is employed to extract features of different scales in channel dimension. To extract the salient features of the image a maximum pooled layer,an upper sampling layer is used beyond convolution layer of 1×1. To compensate for the missing local information in the pooled layer,a 1×1 convolution layer is added to the jump link. Next,the attention mechanism is introduced to learn the correlation of features between different channels,and the fusion of features is realized between channels. Finally,the residual link is used to alleviate over fitting problem while the deep features of the image are extracted. Our gallbladder cancer-based micro-hyperspectral images dataset is derived from the multidimensional common bile duct database produced by Professor Li Qingli’s team of East China Normal University. The database is composed of 880 multi-dimensional image scenes captured from common bile duct tissues of 174 patients. Each microhyperspectral image size is 1 280×1 024×60. The spectral resolution is 2~5 nm and the spectral range is 550~1 000 nm. All images are labeled by expertise. These micro-hyperspectral images of gallbladder carcinoma consists of three different samples:1)background,2)distorted region,and 3)normal region. The background part is organized of cells or blank areas-secreted fat mucus,which can be removed during model training and excluded in the training process. To facilitate the follow-up experiments,the image size is cut to 640×512×60,and four different hyperspectral image datasets are involved in. At the beginning,the spectral validation and principal component analysis(PCA)are used to preprocess the micro-hyperspectral images in order to reduce the interference of the stability of the light source and the noise in the microhyperspectral imaging system to the spectral curves of different tissues in the micro-hyperspectral imaging system. Then, the MSE-ResNet is used to classify the microscopic hyperspectral images of gallbladder carcinoma-relevant pathological sections. Our configuration is equipped with python3. 5. 6 and Keras2. 1. 6 on NVIDIA GeForce RTX 2080 Ti GPU,Intel(R) Xeon(R)CPU E5-2678 v3 CPU. The learning rate is 0. 001,batch size is 16,and dropout rate is 0. 3,as well as the optimization strategy is based on stochastic gradient descent (SGD). To alleviate over fitting problem of the network and improve the generalization ability of the model,three kind of regularization methods are used in MSE-ResNet,which are 1) batch normalization,2)L2 regularization,and 3)dropout regularization. Result The comparison and ablation-related experiments on micro-hyperspectral datasets of gallbladder carcinoma are carried out. Initially,we use several evaluation metrics to evaluate the performance of the MSE-ResNet. The overall classification accuracy,average classification accuracy and kappa coefficient of this model are reached to 99. 619%,99. 581% and 0. 990 of each,which is better than SEResNet and Inception-SE-ResNet. Second,we compare the MSE-ResNet model to other related deep learning and machine learning methods,such as 1D-CNN,ResNet,DenseNet,support vector machine (SVM),and K-nearest neighbor (KNN). The results show that our MSE-Res module can extract the spatial and channel features of micro-hyperspectral images effectively,and classification results can be achieved with less computational cost and better robustness. Our MSEResNet model can be used to learn the features of hyperspectral images automatically and optimize the network parameters through back propagation in comparison with the traditional machine learning methods,which is more beneficial for the classification of micro-hyperspectral images. At last,we compare the micro-hyperspectral image to the traditional RGB image. The experimental results show that the richer band information of the micro-hyperspectral image can improve the model classification results effectively. Conclusion To improve the accuracy of classification of gallbladder cancer,our MSE-ResNet can be focused on the spatial and spectral information of hyperspectral images effectively. It has its potentials for gallbladder cancer-oriented medical diagnosis.
Keywords
hyperspectral image of gallbladder carcinoma multi-scale feature fusion residual network images classification squeeze and excitation(SE)module
|