注意力机制的曲面沉浸式投影系统补偿
雷清桦1,2, 杨婷1,2, 程鹏1,3(1.四川大学视觉合成图形图像技术国家级重点实验室, 成都 610000;2.四川大学计算机学院, 成都 610000;3.四川大学空天科学与工程学院, 成都 610000) 摘 要
目的 沉浸式投影系统已广泛运用于虚拟现实系统之中,然而沉浸式投影系统中的互反射现象严重影响着虚拟现实系统的落地使用。沉浸式投影系统的互反射是指由于投影机光线和屏幕反射光线相互叠加造成的亮度冗余现象,严重影响了投影系统的成像质量和人眼的视觉感受。为此,本文提出一种新的基于互反射通道(inter-reflection channel,IRC)先验和注意力机制的神经网络。方法 IRC先验基于这样一个事实,即大多数受到互反射影响的投影图像都包含一些亮度较高的区域。高亮度区域往往受互反射影响更为严重,而低亮度区域受互反射影响程度较低。根据这一规律,采用IRC先验作为注意力图的监督样本,获取补偿图像的亮度区域信息。同时,为了对投影图像不同区域按影响程度进行差异化补偿,提出一种新的由两个相同子网络构成的补偿网络结构Pair-Net。结果 实验对比了4种现有方法,Pair-Net在ROI(region of interesting)指标分析上取得了明显优势,在人眼感受上有显著的效果提升。结论 本文提出的基于注意力机制的网络模型能够针对不同区域进行差异化补偿,很大程度上消除了互反射影响,提升了沉浸式投影系统的成像质量。
关键词
An attention mechanism based inter-reflection compensation network for immersive projection system
Lei Qinghua1,2, Yang Ting1,2, Cheng Peng1,3(1.Sichuan University National Key Laboratory of Fundamental Science on Synthetic Vision, Chengdu 610000, China;2.College of Computer Science, Sichuan University, Chengdu 610000, China;3.College of Aeronautics and Astronautics, Sichuan University, Chengdu 610000, China) Abstract
Objective Immersive projection system is focused on for the aspects of virtual reality and augmented reality system nowadays. In the context of immersive projection system, the inner-reflection issue is essential to the projection images quality and the fidelity of reality scenes. Inter-reflection refers to brightness redundancy problems derived of overlapping of projector light and screen reflection light in immersive projection system, which severely affects the imaging quality of the projection system. Meanwhile, it is a challenging issue to eliminate optics-based inner-reflection due to the complexity of light transmission in immersive environment. Method A new and simple image prior like inner-reflection channel (IRC) prior and a new attention guide neural network like Pair-Net generate the high-quality inner-reflection compensated projection image in immersive projection system. The IRC prior is a kind of statistic of projection image in immersive projection system. The scenario of most inner-reflection effected projection images are composed of some high intensity pixels. Those high intensity local patches are affected through inner-reflection, which can be used as an attention map to train our compensation net, IRC prior based Pair-Net, a new compensation network, learns the complex reflection and compensation function of immersive projection environment. Result Our experiment demonstrated the improvement in region of interesting (ROI) analysis indicators and human visual perception compared the four existing methods. Pair-Net is capable to learn the complex inner-reflection information and pay attention to the high inner-reflection region. The result of Pair-Net is qualified to the end-to-end projection compensation methods qualitatively and quantitatively. Conclusion Our method illustrates its qualitative and quantitative effectiveness based on significant margin. Immersive projection system have been widely using in those large-scare virtual-reality scene. But, inner-reflection almost exits in all immersive projection system which can heavily decrease the quality of projection image and the fidelity of reality scenes. These challenges often create bottlenecks for generalization of projector system and block the implementation of virtual reality projects. Inner-reflection compensation aims to compensate the projector input image to enhance the projection images quality and lower the effect of inner-reflection. The typical compensation system consists of in-situ projector-camera (pro-cam) pair and a curved screen. The geometric modeling sorts the light transmission and reflection function out. First, light transmission and reflection function in immersive projection environment need to invert a potential large-scale matrix. Next, it is hard for traditional inner-reflection compensation solution to produce high visually quality result due to the mathematical error are inevitable. Finally, current solutions compensate the whole images more and ignore multi-regions based single image intensity issue. A new convolutional neural network (CNN) is prior to photometric compensation domain based photometric compensation algorithm. We facilitated IRC prior and a Pair-Net for inner-reflection compensation. Pair-Net intends to the different patches of image in multiple light intensity immersive projection scenario. The adopted attention mechanisms for different intensity region compensation and use IRC prior to get the attention map. We design Pair-Net as composed of two sub-nets for paying different attention to the higher intensity and lower intensity region in single image. Two auto-encoder sub-net encourages rich multi-level interaction between the camera captured projection image and the ground truth image, and thus capturing the reflection information of the projection screen. Then, the IRC prior yields two sub-net to pay different attention to variance intensity region in immersive projection scenario summary, we first harness an attention guide inner-reflection compensation Pair-Net model in immersive projection system. In addition, the IRC prior is generated the attention map initially.
Keywords
immersive projection system inter-reflection compensation deep learning attention mechanism virtual reality
|