三维重建场景的纹理优化是计算机图形学和计算机视觉等领域的基础任务之一,其目的是优化纹理映射,减小重建几何体和纹理之间的对齐误差,提升重建场景的细节表现。为了对三维重建场景纹理优化算法的现状进行全面研究,本文从传统优化算法和基于深度学习的优化算法两个方面对现有三维重建场景的纹理优化算法进行综述。传统的纹理优化算法一般通过优化相机姿态、校正图像颜色、提高重建几何精度等步骤达到三维场景纹理优化的目的,按照优化方式的不同,主要包括基于图像融合的优化算法、基于图像拼接的优化算法以及纹理与几何联合优化算法,而基于深度学习的优化算法则利用神经网络优化三维场景纹理。同时,本文汇总了常用的三维重建场景纹理优化的数据集与评价指标,并重点讨论了不同数据集和评价指标的特点与用法。此外,本文对现有的各类纹理优化算法进行了定性分析和定量对比,重点阐述了这些算法的原理及优缺点,最后探讨了三维重建场景的纹理优化面临的挑战和发展方向。

Texture optimization is an important task in the field of computer graphics and computer vision, and it plays an essential role in creating realistic 3D reconstructed scenes, which aims to enhance the quality of texture mapping for 3D reconstructed scenes. It has various applications in digital entertainment, heritage restoration, smart cities, virtual/augmented reality, and other fields. To achieve complete texture mapping, a single image is often insufficient, and multiple angle color images are required. By combining these images and projecting them onto the 3D scene, high-quality texture mapping can be achieved, ideally with consistent luminosity across all images. However, the accuracy of 3D reconstruction can be impacted by errors in camera pose estimation and geometry, which result in misaligned projected images. This limitation restricts the use of 3D scenes in various fields, which makes 3D scene texture optimization a crucial task. The texture mapping process involves projecting a 2D image onto a 3D surface to create a realistic representation of the scene. However, this process can be challenging due to the complexity of the 3D data and the inherent inaccuracies in the 3D reconstruction process. Texture optimization algorithms attempt to reduce the errors in texture mapping and improve the visual quality of the resulting scene. Texture optimization algorithms have two main types： traditional and deep learning-based optimization algorithms. Traditional optimization algorithms typically aim to reduce the accumulated error of camera pose and the impact of reconstruction geometry accuracy on texture mapping quality. These algorithms often involve techniques such as image fusion, image stitching, and joint texture and geometry optimization. Image fusion-based optimization algorithms attempt to reduce blurring and artifacts in texture mapping by optimizing the camera pose and adding deformation functions to the images before weighting the color samples of multiple texture images from different views using a mixed weight function. This approach can help improve the visual quality of the scene and reduce the impact of reconstruction errors on texture mapping. Image stitching-based optimization algorithms select an optimal texture image for each triangular slice on the model and then deal with the seams caused by stitching multiple images. This approach can help reduce the impact of reconstruction errors on texture mapping and improve the visual quality of the resulting scene. Joint texture and geometry optimization algorithms further improve the quality of texture mapping by jointly optimizing the camera pose, geometric vertex position, and texture image color. This approach can help minimize the impact of reconstruction errors on texture mapping and achieve more realistic texture mapping. However, traditional optimization algorithms can be computationally expensive due to the complexity of 3D data. To address this challenge, researchers have developed deep learning-based optimization algorithms, which use neural networks to optimize 3D scene textures. These algorithms can be further classified into convolutional neural network-based optimization algorithms, generative adversarial network （GAN）-based optimization algorithms, neural textures, and text-driven diffusion model optimization algorithms. Convolutional neural network-based optimization algorithms use deep neural networks to learn the texture features of the input images and generate high-quality texture maps for 3D reconstructed scenes. GAN-based optimization algorithms use a GAN to generate high-quality texture maps that are visually indistinguishable from real images. Neural textures use a neural network to synthesize new textures that can be applied to 3D reconstructed scenes. Text-driven diffusion model optimization algorithms use a diffusion model to optimize the texture of large missing 3D scenes based on a given text description. Deep learning-based optimization algorithms have shown promising results in improving the visual quality of texture mapping and reducing computational costs. They can effectively combine prior knowledge with neural networks to infer the texture of large missing 3D scenes. This feature not only reduces computation but also ensures texture details and greatly enhances visual realism. In addition to texture optimization algorithms, various datasets and evaluation metrics have been developed to evaluate the performance of these algorithms. The commonly used datasets include the bundle fusion dataset and the ScanNet dataset. The evaluation metrics used to evaluate the performance of texture optimization algorithms include peak signal-to-noise ratio, structural similarity index, and mean squared error. In conclusion, texture optimization of 3D reconstructed scenes is a challenging research topic in the fields of computer graphics and computer vision. Traditional and deep learning-based optimization algorithms have been developed to address this challenge. The future trend of texture optimization is likely to be deep learning-based optimization algorithms, which can effectively reduce computational costs and enhance visual realism. However, many challenges and opportunities exist in this field, and researchers need to continue exploring new approaches to improve the quality of texture mapping for 3D reconstructed scenes.