自适应卷积约束与全局上下文推理的墓室壁画修复
吴萌1, 郭歌1, 孙增国2, 路智勇3, 张倩文1(1.西安建筑科技大学;2.陕西师范大学;3.陕西历史博物馆) 摘 要
目的:墓室壁画作为地下文物,由于环境湿度、地仗沉降等因素,局部区域出现了脱落、裂缝、霉变等多种病害,导致画面部分缺失。但现有深度学习的修复方法通常在单一维度或固定区域进行信息重建,无法充分捕获稀疏的壁画特征和修复多样化的复杂病害,修复时会出现内容缺失、结构错乱等问题。对此,本文提出了一种自适应卷积约束与全局上下文推理的墓室壁画修复。方法:该方法基于端到端的编码器-解码器架构,首先设计多尺度增强卷积模块从频域和空域联合分析图像特性来充分捕获全局结构和局部纹理;同时在修复路径中加入融合差分卷积的增强激活单元来引入边缘先验信息,提高模型的绘制精度。其次,考虑到纹理和结构在绘制过程中的模式差异,在编码器-解码器间设计基于注意力交互引导的多尺度特征聚合模块,来加强全局稀疏信息的表征能力和相关性,并自适应选择增强有效特征。此外,为了获得真实准确的结果,在特征传递过程中利用自动掩码更新迭代来预测复杂缺失信息,引导解码器精确绘制多样化的损伤区域。结果:本文采用客观评价指标:峰值信噪比(peak signal-to-noise ratio,PSNR)、结构相似性(structural similarity index,SSIM)、学习感知图像块相似度(learned perceptual image patch similarity,LPIPS)在章怀太子墓“马球图”数据集上进行三类模拟病害和真实病害修复实验,并与最新的6种主流方法进行了比较。实验结果表明本文方法修复的壁画图像在主观视觉和客观评价上均有明显的提升。如相较于指标排名第二的模型,对于随机缺失区域的壁画修复,峰值信噪比和结构相似性的均值分别达到31.7602dB和0.9577,各指标的样本均值分别提升了2.3653dB,0.0128,12.75%。结论:本文提出的图像修复模型可以有效修复多种复杂病害,可为手工绘制专家的物理修复提供参考,这进一步表明了该方法在保护和传承数字文化遗产的有效性和适用性。
关键词
Tomb mural inpainting with adaptive convolutional constraints and global context inference
() Abstract
Objective:As an important cultural heritage of ancient civilization, with the changes of the years, due to factors such as environmental humidity and ground settlement, the local areas of the murals in the tomb have fallen off, cracks, mud spots, mildew and other diseases, which seriously affect the sustainable development of the protection of mural cultural relics and the activities of appreciation, cultural creativity and cultural dissemination. Due to the influence of underground environmental factors, in order to protect the precious mural relics, researchers usually adopt the means of block excavation to transfer the mural paintings in the tomb and restore them, and the restoration of murals in the past requires professional restorers to draw manually, which requires a high level of manual operation, and the repair cycle is lengthy and inefficient. Therefore, in the face of the complex semantic environment and the lack of diverse information, the restoration method based on deep learning has been gradually applied to the information reconstruction of murals to provide scientific and technological protection for murals. However, most of the existing methods usually perform restoration in a single dimension or a fixed area, which cannot fully capture the sparse mural features and repair multiple complex diseases at the same time, resulting in semantic inconsistencies or incoherent structural results. In order to solve the above problems, this paper proposes a tomb mural inpainting model with adaptive convolutional constraints and global context inference, which can repair various types of damage and diseases, and produces a rich database of digital cultural heritage. Methods:Based on the end-to-end encoder-decoder architecture, the model firstly designs a multi-scale enhanced convolution (MEConv) module in the encoder path for content constraints, extracts different features of the image from the frequency domain and spatial domain at the same time to complement each other, and the enhanced activation unit fused with differential convolution is added to the repair path to introduce edge prior information, so as to correct the adaptive multi-scale feature mapping, which can more effectively capture the global and local latent feature information. Secondly, considering the differences in the patterns of texture and structure in the mural painting process, a multi-scale feature aggregation (MSFA) module was added between the encoder and the decoder, and through the fusion of multi-scale feature components and the dynamic selection of attention mechanism, the adaptive selection and enhanced effective information of the global context were strengthened, and the representation and generalization ability of the original feature map were strengthened, and the drawing accuracy was improved. In addition, in order to obtain true and accurate results, in the process of feature transfer, the model compensates for the difference between the underlying and high-level features through automatic mask update and jump connection in each layer, and guides the decoder to accurately draw the missing content. Results:In order to verify the effect of network disease repair, this paper used the objective evaluation indexes: peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and learned perceptual image patch similarity (LPIPS) to carry out three types of disease remediation experiments on the constructed polo mural dataset, and compared them with the latest four mainstream methods. The experimental results show that the images restored by this method have obvious improvement in subjective vision and objective evaluation, and the content and style of the generated murals are closer to those of the real murals. For example, compared with the second-ranked algorithm, the mean values of each index were increased by 0.1432dB, 0.0167 and 2.17% respectively for mural restoration in the epitaxial damage area. For the mural repair in the irregular damage area, the mean values of each index increased by 0.4132dB, 0.0304 and 15.11%, respectively. For the mural restoration in the random damage area, the mean values of each index increased by 2.3653dB, 0.0128 and 12.75%, respectively. Conclusion:The image inpainting model proposed in this paper can not only fully capture the latent features of different characteristics of the image, but also capture the long-term contextual semantic features in the deep features. The model can effectively deal with a variety of complex mural diseases, and restore images with consistent semantics, rich details, complete content and natural coherence, providing a feasible solution for the digital restoration and display of murals. In order to restore the real damaged mural images, in the future work, we will collect more high-definition tomb murals, and fully consider the mural painting and historical background of different periods and different years, so as to facilitate the model to learn more accurate and comprehensive mural content and style, and improve the restoration performance of the model.
Keywords
|