Current Issue Cover
面向非均质图像去雾的解耦合三阶段增强网络

刘春晓, 胡鹏靖, 厉世昌, 王成骅, 凌云(浙江工商大学)

摘 要
目的 在雾霾环境下拍摄的图像通常具有结构对比度较低、细节信息模糊和颜色饱和度失真等特点。目前的去雾算法,虽然已经能较好的处理均质雾霾图像,但是对于非均质雾霾图像的去雾能力仍较差。为此,本文提出了一种面向非均质雾霾图像去雾的解耦合三阶段增强网络。方法 通过颜色空间变换将输入图像解耦为亮度、饱和度和色度三个通道之后,该算法首先通过对比度增强模块增强亮度图的对比度,使去雾结果具有更清晰的结构和细节信息;然后,通过饱和度增强模块增强图像的饱和度,使去雾结果具有更鲜艳的颜色;最后,使用颜色矫正增强模块对总体颜色进行微调,使去雾结果更符合人眼视觉感知。特别地,我们在饱和度增强模块中设计了一个雾霾密度编码矩阵,通过计算亮度图在对比度增强前后的梯度差异,估计出雾霾图像的雾霾密度信息,为饱和度增强模块提供指导,以保证饱和度恢复的准确性。结果 在3个数据集上与14种方法进行了对比实验,在NHD数据集上得到了最优结果,相比于性能第2的模型,平均峰值信噪比提升了8.5dB,平均结构相似性提升了0.12;在Real-World数据集中,我们的感知雾密度预测值为0.47,雾密度估计值为0.21,均处于前列;在SOTS数据集中,我们的平均峰值信噪比为16.52dB,平均结构相似性为0.80,在人眼感知效果方面不输于已有方法。结论 实验结果表明,本文所提出的算法对于非均质雾霾图像具有优秀的处理能力,可以有效地去除图像的雾霾并还原出雾霾图像的真实细节信息和颜色。
关键词
Decoupled triple-stage enhancement network for non-homogeneous image dehazing

LIU CHUN XIAO, HU PENG JING, LI SHI CHANG, Wang Cheng Hua, LING YUN()

Abstract
Objective The absorption or scattering effect of microscopic particles in the atmosphere, such as aerosols, soot, haze, etc., will reduce the image contrast, blur the image details, and cause the color distortion. These problems can lead to a decrease in the accuracy of subsequent advanced computer vision tasks, such as object detection, image segmentation, etc. Therefore, the task of image dehazing has attracted more and more attention, various image dehazing methods have been proposed. The ultimate goal of image dehazing is to recover a haze-free image from the input hazy image. At present, existing image dehazing algorithms can be divided into two categories, one is the traditional dehazing algorithms based on image prior, and the other is the image dehazing algorithms based on deep learning. The image priori-based dehazing algorithm uses the prior information and empirical rules of the image itself to estimate the transmittance map and atmospheric light value, and utilizes the atmospheric scattering model to realize the image dehazing process, which can improve the contrast of the image to a certain extent, but it is easy to lead to excessive enhancement or color distortions in the dehazed results. Driven by a large number of image data, the image dehazing algorithm based on deep learning can flexibly learn the mapping from hazy image to haze-free images by directly constructing an efficient convolutional neural network, and obtain the dehazed effects with better generalization performance and better human visual perception. However, due to the existence of domain differences, it is usually difficult for the image dehazing algorithm trained on the synthesized homogeneous haze dataset to achieve satisfactory results on the heterogeneous hazy images in the real world. Method Since haze will reduce the contrast of the image and make the image looks blurring, we train the network (i.e., the contrast enhancement module) with the brightness map of the hazy image and the brightness map corresponding to the clear image as the training image pairs, which effectively enhances the contrast of the brightness map and obtains the brightness enhancement map with clear image structure and details. Furthermore, we calculate the gradient differences of the brightness maps before and after the contrast enhancement process, and estimate the haze density information in the hazy images for the guidance of saturation enhancement of the hazy images. Therefore, we propose an end-to-end decoupled triple-stage enhancement network for the heterogeneous haze dehazing task, which decouples the input hazy image with color space conversion into three channels, i.e. brightness, saturation, hues. Our algorithm first enhances the contrast of the brightness map through the contrast enhancement module, so that the dehazed result holds clear structure and detail information, then enhances the saturation channel of the image through the saturation enhancement module, so that the dehazed result takes on more vivid color, and finally the color correction and enhancement module is used to fine-tune the overall color of the image, so that the final dehazed result will be more in line with the human visual perception. In particular, we design a haze density coding matrix in the saturation enhancement module, and estimate the haze density information of the hazy image by calculating the gradient differences of the brightness maps before and after the contrast enhancement process, so as to provide guidance for the saturation enhancement module to ensure the accuracy of saturation recovery. Due to the superiority of U-net network structure for the image enhancement tasks, we choose U-net as the backbone network of our contrast enhancement module and saturation enhancement module, and obtain multi-scale information of images through the encoder and decoder structure for better dehazing results. For the color correction and enhancement module, since we only need to fine-tune the previously enhanced image results, we only use a simple network with convolutional layers and skip connections to prevent the loss of image information with up-sampling and down-sampling operations. Result Compared with the second-best model in performance, the average peak signal-to-noise ratio (PSNR) is increased by 8.5dB and the average structural similarity is increased by 0.12. And, our perceived fog density prediction value (FADE) is 0.47 and the estimated haze density (DezHaz) is 0.21 in the Real-World dataset, both of which are in the forefront. In the SOTS dataset, our average peak signal-to-noise ratio is 16.52dB and the average structural similarity is 0.80, which are comparable to the existing algorithms in terms of human visual perception. Conclusion Through a series of subjective and objective experimental comparisons, the experimental results show that our algorithm has excellent processing ability for non-homogeneous hazy images, and can effectively restore the real details and colors of the hazy images.
Keywords

订阅号|日报