Current Issue Cover
融合注意力机制和上下文信息的微光图像增强

赵兴运1, 孙帮勇1,2(1.西安理工大学印刷包装与数字媒体学院, 西安 710048;2.中国科学院西安光学精密机械研究所光谱成像技术重点实验室, 西安 710119)

摘 要
目的 微光图像存在低对比度、噪声伪影和颜色失真等退化问题,造成图像的视觉感受质量较差,同时也导致后续图像识别、分类和检测等任务的精度降低。针对以上问题,提出一种融合注意力机制和上下文信息的微光图像增强方法。方法 为提高运算精度,以U型结构网络为基础构建了一种端到端的微光图像增强网络框架,主要由注意力机制编/解码模块、跨尺度上下文模块和融合模块等组成。由混合注意力块(包括空间注意力和通道注意力)引导主干网络学习,其空间注意力模块用于计算空间位置的权重以学习不同区域的噪声特征,而通道注意力模块根据不同通道的颜色信息计算通道权重,以提升网络的颜色信息重建能力。此外,跨尺度上下文模块用于聚合各阶段网络中的深层和浅层特征,借助融合机制来提高网络的亮度和颜色增强效果。结果 本文方法与现有主流方法进行定量和定性对比实验,结果显示本文方法显著提升了微光图像亮度,并且较好保持了图像颜色一致性,原微光图像较暗区域的噪点显著去除,重建图像的纹理细节清晰。在峰值信噪比(peak signal-to-noise ratio,PSNR)、结构相似性(structural similarity,SSIM)和图像感知相似度(perceptual image patch similarity,LPIPS)等客观指标上,本文方法较其他方法的最优值分别提高了0.74 dB、0.153和0.172。结论 本文方法能有效解决微光图像存在的曝光不足、噪声干扰和颜色不一致等问题,具有一定应用价值。
关键词
An attention mechanism and contextual information based low-light image enhancement method

Zhao Xingyun1, Sun Bangyong1,2(1.Faculty of Printing, Packaging and Digital Media, Xi'an University of Technology, Xi'an 710048, China;2.Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China)

Abstract
Objective Low-light images capturing is common used in the night or dark scenario. In a low-light imaging scenario, the number of photons penetrating the lens is tiny, resulting in poor image quality, poor visibility, low contrast, more noise, and color distortion. Low-light images constrain many subsequent image processing tasks on the aspects of image classification, target recognition, intelligent monitoring and object recognition. The current low-light level image enhancement technology has mainly two challenging issues as following:1) space-independent image brightness; 2) non-uniform noise. The spatial feature distribution of real low-light images is complex, and the number of penetrating photons varies greatly at different spatial positions, resulting in strong light-related variability in the image space. Existing deep learning methods can effectively improve the illumination characteristics in artificially generated datasets, but the overall image visibility and the enhancement effect of underexposed areas need to be improved for the space-restricted low illumination. For instance, de-noising will ignore some image details before image enhancement. It is difficult to reconstruct high-noise pixel information, and de-noising will easily lead to blurred images after enhancement. Our attention mechanism and contextual information based low-light image enhancement method can fully enhance the image to suppress potential noise and maintain color consistency. Method Our demonstration constructs an end-to-end low-light enhancement network, using U-Net as the basic framework, which mainly includes channel attention modules, encoders, decoders, cross-scale context modules and feature fusion modules. The backbone network is mainly guided by mixed attention block. The input is the low-light image to be enhanced, and the output is an enhanced high-quality noise-free color image of the same size. After the low-light enhancement network extracts the shallow features of the input original low-light image, it first uses the channel attention module to learn the weight of each channel, and assigns more weight to the useful color information channel to extract more effective image color features. Then, the channel weight and the original low-light image are used as the input of the encoder, and the semantic features of the image are extracted by the mixed attention block. Mixed attention block extracts input features from spatial information and channel information respectively. It is beneficial to restore the brightness and color information of the image and noise suppression via the location of noise in the image and the color features of different channels. In the decoder part, the de-convolution module restores the semantic features extracted by mixed attention block to high-resolution images. In addition, a cross-scale context module is designed to fuse cross-scale skip connections based on the skip connections performed on the corresponding scale features of conventional encoders and decoders. Result In respect of qualitative evaluation, our method is slightly higher than the reference image in fitting the overall contrast of the image, but it is better than all other contrast methods in terms of color details and noise suppression. In terms of quantitative evaluation indicators of image quality, including peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and perceptual image patch similarity (LPIPS). The methods proposed in this paper have also achieved excellent results, which are 0.74 dB, 0.153 and 0.172 higher than the optimal values of other methods. The results show that our proposed method can enhance the brightness of the image and maintain the consistency of the image color, better remove the influence of noise in the dark area of the image, and naturally retain the texture details of the image. Conclusion Our method has its priority in brightness enhancement, noise suppression, image texture structure and color consistency.
Keywords

订阅号|日报