Current Issue Cover
迁移学习在医学图像分类中的研究进展

黎英, 宋佩华(南宁师范大学广西人机交互与智能决策重点实验室, 南宁 530001)

摘 要
医学影像作为医疗数据的主要载体,在疾病预防、诊断和治疗中发挥着重要作用。医学图像分类是医学影像分析的重要组成部分。如何提高医学图像分类效率是一个持续的研究问题。随着计算机技术进步,医学图像分类方法已经从传统方法转到深度学习,再到目前热门的迁移学习。虽然迁移学习在医学图像分类中得到较广泛应用,但存在不少问题,本文对该领域的迁移学习应用情况进行综述,从中总结经验和发现问题,为未来研究提供线索。1)对基于迁移学习的医学图像分类研究的重要文献进行梳理、分析和总结,概括出3种迁移学习策略,即迁移模型的结构调整策略、参数调整策略和从迁移模型中提取特征的策略;2)从各文献研究设计的迁移学习过程中提炼共性,总结为5种迁移学习模式,即深度卷积神经网络(deep convolution neural network,DCNN)模式、混合模式、特征组合分类模式、多分类器融合模式和二次迁移模式。阐述了迁移学习策略和迁移学习模式之间的关系。这些迁移学习策略和模式有助于从更高的抽象层次展现迁移学习应用于医学图像分类领域的情况;3)阐述这些迁移学习策略和模式在医学图像分类中的具体应用,分析这些策略及模式的优点、局限性及适用场景;4)给出迁移学习在医学图像分类应用中存在的问题并展望未来研究方向。
关键词
Review of transfer learning in medical image classification

Li Ying, Song Peihua(Guangxi Key Laboratory of Human-machine Interaction and Intelligent Decision, Nanning Normal University, Nanning 530001, China)

Abstract
Medical image classification is a key element of medical image analysis. The method of medical image classification has been evolving in deep learning and transfer learning. A large number of important literatures of medical image classification are analyzed based on transfer learning. As a result, three transfer learning strategies and five modes of medical image classifying are summarized. The transfer learning modes are constructed based on general characteristics which extracted from designed transfer learning processes theoretically. The relevance of the transfer learning strategies and the modes is illustrated as well. These transfer strategies and modes illustrated the application of transfer learning in this field from a higher level of abstraction. The applications, advantages, limitations of these transfer learning strategies and modes are analyzed. The transfer learning in the context of medical image feature extraction and classification is the model-based transfer learning. Most of the migrated models are deep convolution neural network (DCNN). High classifying efficiency is obtained due to ImageNet (a large public image database) training results. To migrate model from the source domain to the target domain, the model needs to be adapted to the tasks of the targeted domain. From the important literature of medical image classification, three transfer model adaptive strategies were sorted out. They are structure fit, parameter option and migrated model based features extraction. The strategies of fitted structure are to modify the structure of the migrated model. Layers can be deleted or added as needed. These layers can be as convolution layer, full connection layer, feature smoothing layer, feature extracting layer, and batch normalizing layer, respectively. The optional strategies of parameter are to re-train the migrated model to adjust the model parameters via using the target domain data. Both the parameters of convolution layer and the parameters of full connection layer can be fitted. The migrated-model-based strategy of features extraction is only used for feature extraction. Image features can be extracted from the convolution layer and the full connection layer. Five transfer learning modes were sorted out based on the literature of medical image classification. They are the DCNN mode, the hybrid mode, the mode of fused feature and classification. The mode of multi-classifier fusing and the mode of transferring are conducted two times all. For instance, the fitting strategies of structure and parameters of convolution layer are used in the DCNN mode to get more accurate features. The transfer learning modes can basically cover all kinds of transfer learning processes in medical image classification researches. The DCNN mode is to use a DCNN to complete image feature extraction and classification. The hybrid model is composed of a DCNN and a traditional classifier. The former is used for feature extracting and the latter is used for classifying. It has the advantage of DCNN feature extraction and advantage of traditional classified classifier. The mode of integrated feature and classification is composed of feature extraction methods and a classifier. Feature extracting methods can be a DCNN or manual features. The mode of multi-classifier fusing is composed of multiple classifiers. The final classifying result is obtained based on multiple classified integrating results. The integrating result is more reliable than the result from single classifier mode (i.e., the DCNN mode and the hybrid mode). In the twice mode of transferring, the initial model is trained in the source domain, then it is migrated to the temporary targeted domain to be trained for the second time, and finally it is migrated to the final targeted domain to be trained for the third time. Compared to first transfer, this mode priority is that the triple migrated model accumulates more knowledge of training. The transfer-learning based issues and of medical image classification potentials are illustrated as below:1) It is difficult to pick an efficient transfer learning algorithm. Due to the diversity and complexity of medical images, the generalization capability of transfer learning algorithm is to be strengthened. Manual option for a specific image classifying task is often based on continuous high computational cost tests. A low cost automatic transfer learning algorithm is a challenging issue. 2) The modification of transfer model and the setting of mega parameters are lack of theoretical mechanism. The structure and parameters of the migrated model can only be modified through the existing experiences and continuous experiments, and the setting of mega parameters is not different, which lead to the low efficiency of transfer learning. 3) It is challenged to classify rare disease images due to pathological image data samples of rare diseases. The antagonistic transfer learning can generate and enhance target domain data. The heterogeneous transfer learning yield to transfer the knowledge of different modalities or the source domain to target domain. Therefore, the two methods of transfer learning can be further developed in image classification of rare diseases.
Keywords

订阅号|日报