Current Issue Cover
甲状腺超声影像的元优化多级对抗域适应网络

应翔1,2,3, 刘振1,2,3, 朱佳琳4, 姜汉5, 张瑞璇1,2,3, 高洁1,2,3(1.天津大学智能与计算学部, 天津 300350;2.天津市认知计算与应用重点实验室, 天津 300350;3.天津市先进网络技术与应用重点实验室, 天津 300350;4.天津医科大学肿瘤医院, 天津 300060;5.贝式计算(天津)信息技术有限公司, 天津 300456)

摘 要
目的 计算机辅助诊断是临床诊断中一种重要的辅助手段。然而在多机型超声影像的应用现状中,单一深度卷积神经网络面临难以从不同数据源中提取样本特征的问题,导致模型在区分多源数据方面性能欠佳。为提升单一深度模型在多源数据的泛化能力,本文提出一种无监督域自适应网络。方法 将深度对抗域适应方法应用于多源甲状腺超声影像分类任务,通过生成对抗思想提取源域图像与目标域图像的域不变特征,提出一种多级对抗域自适应网络(multi-level adversarial domain adaptation network,MADAN)。将元优化(meta-optimized)策略引入对抗域适应的学习中,将域对齐目标和样本分类目标以协调的方式联合优化,提升了模型对无标记目标域数据的分类性能。结果 在包含4种域的甲状腺超声影像数据集上实验,与7种经典域自适应方法比较。实验结果表明,MADAN在全部迁移任务中取得90.141%的目标域样本平均分类准确率,优于残差分类网络和多种经典域自适应分类网络。融合元优化训练策略后的MADAN在目标域的测试平均准确率提升约1.67%。结论 本文提出的元优化多级对抗域适应网络一方面通过多级对抗学习进行图像域不变特征的提取,另一方面使用元优化方式改进模型训练过程的优化策略,将带有人工标记的源域信息有效迁移至目标域,提升了单一模型对于不同域数据的泛化性能。
关键词
Meta-optimized multi-adversarial domain adaptation for thyroid ultrasound image

Ying Xiang1,2,3, Liu Zhen1,2,3, Zhu Jialin4, Jiang Han5, Zhang Ruixuan1,2,3, Gao Jie1,2,3(1.College of Intelligence and Computing, Tianjin University, Tianjin 300350, China;2.Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin 300350, China;3.Tianjin Key Laboratory of Advanced Networking, Tianjin 300350, China;4.Medical University Cancer Institute and Hospital, Tianjin 300060, China;5.OpenBayes (Tianjin) IT Co., Ltd., Tianjin 300456, China)

Abstract
Objective Artificial intelligence based (AI-based) medical clinical diagnosis technique has been developing in recent years. It can alleviate the problems of medical image analysis in China via deep networks modeling and medical image data analysis, such as the shortage of expertises, the imbalance of urban and rural medical resources allocation, and the the imaging accuracy issues. However, clinical-aided are often linked to multi-model data with different characteristics distribution in different hospitals. Therefore, improving the generalization and stability of the cross-model medical diagnosis-consistent model is required for quick response intensively. In order to alleviate the domain shift existing in the ultrasound imaging field, the unsupervised domain adaptation method can be as the one of the most concerning methods at present. It can avoid the manual labeling of ultrasound image data of various models, a single model can be learnt to adapt to the target domain sample set with data deviation through the labeled source domain sample set, which improves the generalizability of convolutional neural network (CNN) to a certain extent. However, current unsupervised domain adaptation research has some challenging constraints, such as poor feature extraction and inconsistent optimization of domain fusion and sample classification. In view of the limitations in related to domain adaptation network, we develop an intergrated domain adaptation network, which focuses on the under-expressed feature of nodular region features in thyroid ultrasound images. This research is aimed to enhance the fusion of source domain features space and target domain features space. Method In this study, a new domain adaptation network is constructed called based on domain-adversarial training of neural networks (DANN), called multi-level adversarial domain adaptation network (MADAN). In the training process, we first build a three-layer generator and discriminator structure according to the transition from general to special features, which can obtain more semantic information in the image. To ensure the coordination of the two optimization processes, the sample classification task and the domain fusion task are both implemented via a meta-optimization strategy. Furthermore, to enhance the global geometric features of diseased tissue regions, we embed a self-attention module in the adversarial structure. Our dataset is composed of 6 849 images-selected from ultrasound images in 4 different models, namely P (Philips1), T(Toshiba), F (GE), and U (Philips2). Our domain adaptation network is carried out in the PyTorch toolbox. The input image is resized to 224 × 224 pixels for training, the momentum parameter is set to 0.9, and the learning rate is set to 0.001. The stochastic gradient descent (SGD)-related learning process is accelerated using an NVIDIA GTX TITAN RTX device, which takes approximately 10 hours in 50 000 iterations. It is possible to obtain more effective samples in the context to benign and malignant labels of the source domain, the target domain during the training process, and the labels of the target domain data are not required at all. The training process performs the supervised classification task of the source domain dataset and the domain alignment task. In the testing process, the feature extractor and classifier fitted are used to verify the classification accuracy of the target domain data. In addition, we use t-distributed stochastic neighbor embedding (t-SNE) to visualize the learned data features, which demonstrates the effectiveness of our network further on the feature-fused of different datasets. Result We compared our network with 8 domain adaptation networks, including domain adaptation tasks based on 9 multi-directions on 4 domains: P→T, P→U, P→F, T→P, T→U, T→F, U→P, U→T, and U→F. The evaluation metrics are focused on the accuracy of the target domain data and the t-SNE visualization results, and the ablation experiments is conducted to demonstrate the performance and GPU memory cost of the proposed method. Experimental results show that the MADAN is optimized with an average classification accuracy of 90.141% on the multi-model thyroid ultrasound dataset through 9 transfer tasks. After introducing meta-optimization, the best classification accuracy is achieved in PU, PF, TP, TF, UP, and UF tasks respectively, and the average accuracy of all tasks is improved about 1.67%. The results of the ablation experiments illustrates that the domain adaptation, multi-adversarial structure, and self-attention modules are improved 16.238%, 20.284%, and 18.622%, respectively. The t-SNE visualization images are illustrated that the feature space of the data samples has been preliminarily fused after the multi-level adversarial domain adaptation, and the sample points of the same category are basically aggregated. The multi-adversarial domain adaptation method can achieve fusion results better via the improved meta-optimization strategy. Additionally, the comparative visualization analysis of the heat map verifies the effectiveness of the self-attention module for global geometric feature extraction of the lesion tissue. Conclusion Our research is focused on meta-optimized multi-Adversarial domain adaptation network, including a multi-level generative adversarial structure and a meta-optimization strategy. The experimental results show that the proposed method is performed well in the transfer task of multi-model medical thyroid ultrasound datasets and has better classification results in a completed unsupervised setting.
Keywords

订阅号|日报