Current Issue Cover
残差密集结构的东巴画渐进式重建

蒋梦洁, 钱文华, 徐丹, 吴昊, 柳春宇(云南大学信息学院, 昆明 650504)

摘 要
目的 东巴画具有内容丰富、线条疏密相间、色彩多样的独特艺术风格,将现有针对自然图像的超分辨率算法直接应用于低分辨率东巴画时,对东巴画线条、色块以及材质的重建效果不理想。为了有效提高东巴画数字图像分辨率,本文针对东巴画提出超分辨率重建方法。方法 首先,针对东巴画图像包含丰富高频信息的特点搭建重建网络,网络整体结构采用多级子网络级联方式渐进地重建出高分辨率东巴画,多级子网络标签共同指导重建,减少了东巴画图像在上采样过程中高频细节的丢失,每一级子网络内部均包含浅层特征提取模块和以残差密集结构为核心的深层特征提取模块,分别提取东巴画不同层次的特征进行融合,改善了卷积层简单的链式堆叠造成的特征丢失。其次,为了进一步提升重建东巴画的视觉质量,在像素损失的基础上引入感知损失和对抗损失进行对抗训练。最后,为了使网络对东巴画图像特征的学习更具针对性,本文自建东巴画数据集用于网络训练。结果 实验结果表明,在本文东巴画测试集set20上,当上采样因子为8时,相较于Bicubic(bicubic interpolation)、SRCNN(super-resolution convolutional neural network)、Srresnet(super-resolution residual network)和IMDN(information multi-distillation network)方法,本文算法的峰值信噪比分别增加了3.28 dB、1.80 dB、0.23 dB和0.36 dB,重建东巴画的主观视觉质量也得到了更好的结果。结论 本文提出的超分辨率网络模型能有效提高低分辨率东巴画的分辨率和清晰度,并且具有普适性,亦可采用其他数据集进行训练以拓展应用范围。
关键词
Gradual model reconstruction of Dongba painting based on residual dense structure

Jiang Mengjie, Qian Wenhua, Xu Dan, Wu Hao, Liu Chunyu(School of Information Science and Engineering, Yunnan University, Kunming 650504, China)

Abstract
Objective Dongba painting is an essential part of Na’xi culture in China. It is the fundamental medium to develop Dongba culture in art theory, natural aesthetics, religion and history. However, the current low resolution digital image of Dongba painting has affected the application, inheritance and development of Dongba culture. Super-resolution reconstruction technology is the process of recovering high-resolution image from low-resolution image. Because Dongba painting has a unique artistic style, compared with natural images, there are no dimension, distance and depth factors, and there is no light and shadow effect via natural light. While the existing super-resolution algorithm for natural images applied to Dongba painting straitforward, the reconstruction effect of lines, color blocks and materials of Dongba painting is not ideal. Therefore, the super-resolution reconstruction of Dongba painting is adopted and high-resolution Dongba painting obtained via the reconstruction of low-resolution Dongba painting. Method First, the Dongba painting dataset for network training is constructed, which makes the learning of Dongba painting image characteristics more targeted. The data set contains 298 high-definition Dongba paintings, and each Dongba painting has one dimension with a resolution of 2 K at least. Among them, 278 paintings are used for training, 20 paintings are used for testing, and the training set is randomly cropped for data enhancement. Next, in accordance with the characteristics of Dongba painting image which is rich in high frequency information, a reconstruction network is built and named Dongba super-resolution network (DBSRN): the overall structure of the network uses multi-level sub network cascade to gradually reconstruct high-resolution Dongba painting. During the feedforward process of a large-scale reconstruction, multiple intermediate Super-resolution predictions were produced. The final reconstruction results are constrained by multiple intermediate prediction values, so as to reconstruct Dongba paintings of different scales from small to large gradually. The reconstruction results of each sub network and the corresponding scale tags are calculated at the pixel level, and the tags of different scales jointly guide the reconstruction. The loss of high-frequency details is reduced in the up sampling process of Dongba painting image. To extract and fuse features at different levels, each level of super-resolution sub-network has a shallow feature extraction module, a deep feature extraction module, and a global feature fusion module. In this way, the disappearance of features with the deepening of network can be avoided. The extracted feature maps are input to the up-sampling module for reconstruction. The residual dense structure, which combines residual connection and dense connection, can effectively enhance feature reuse and slow down gradient disappearance. The demonstrated algorithm takes the residual dense structure as the core in deep feature extraction module, extracts the deep feature of Dongba painting for fusion, reduce the feature loss caused by simple chain stacking in the convolution layer; At the end, added-discriminator, perception loss and adversarial loss for adversarial training are implemented to improve the visual quality of Dongba painting based on pixel level loss. This network is named DBGAN (Dongba generative adversarial network). Result This illustration demonstrate the Dongba paintings reconstructed in this paper get better results in both subjective visual quality and objective indicators, compared to bicubic interpolation (Bicubic), super-resolution convolutional neural network (SRCNN), super-resolution residual network (Srresnet) and information multi-distillation network (IMDN).The peak signal to noise ratio (PSNR)/structural similarity index (SSIM) in DBSRN reach 33.46 dB and 0.911 2 when upsampling factor is 2, 28.54 dB and 0.776 2 when upsampling factor is 4, and 24.61 dB and 0.643 0 when upsampling factor is 8. The objective indicators of DBSRN are improved to varying degrees compared with Bicubic, SRCNN, and Srresnet. Compared with Srresnet, when the upsampling factor is 2, PSNR and SSIM are increased by 0.10 dB and 0.000 8, respectively, when the upsampling factor is 4, they are increased by 0.18 dB and 0.003 2, and when the upsampling factor is 8, they are increased by 0.23 dB and 0.004 4. DBGAN improves the clarity and fidelity of the reconstruction results further and reconstruct Dongba paintings with more edge and texture details. Conclusion This super-resolution universal network model can improve the resolution and clarity of low-resolution Dongba paintings effectively.
Keywords

订阅号|日报