联合形态学滤波和卷积稀疏编码的图像去雨
黄淑英1, 许亚婷2, 杨勇1, 管巨伟3(1.天津工业大学计算机科学与技术学院, 天津 300387;2.江西财经大学软件与物联网工程学院, 南昌 330032;3.江西财经大学信息管理学院, 南昌 330032) 摘 要
目的 图像去雨技术是对雨天拍摄图像中雨纹信息进行检测和去除,恢复目标场景的细节信息,从而获得清晰的无雨图像。针对现有方法对雨纹信息检测不完全、去除不彻底的问题,提出一种联合自适应形态学滤波和多尺度卷积稀疏编码(multi-scale convolution sparse coding, MS-CSC)的单幅图像去雨方法。方法 考虑雨纹信息的形状结构特点,构造一种自适应形态学滤波器来滤除有雨图像中的雨纹信息,获得包含图像自身纹理的低频成分;利用全变分模型正则化方法来增强低频成分的纹理信息,并利用有雨图像减去低频成分获得包含雨纹信息的高频成分;针对高频成分,根据雨纹的方向性提出一种基于方向梯度正则化的MS-CSC方法来重构高频成分,并通过迭代求解获得包含精确雨纹的高频成分,即雨层;利用有雨图像减去雨层得到最终的去雨图像。结果 为验证本文方法的有效性,与一些主流的去雨方法进行实验比较。实验结果表明,本文方法在模拟数据集上的平均峰值信噪比(peak signal-to-noise ratio, PSNR)和平均结构相似度(structural similarity, SSIM)指标分别提高了0.95和0.005 2,能得到较好的主观视觉效果和客观评价,特别是在真实数据集上具有明显优势。结论 本文方法可有效去除雨纹,同时可保留更多的原始图像纹理信息,经证明是一种有效的去雨算法。
关键词
Image deraining algorithm based on morphological filtering and convolution sparse coding
Huang Shuying1, Xu Yating2, Yang Yong1, Guan Juwei3(1.School of Computer Science and Technology, Tiangong University, Tianjin 300387, China;2.School of Software and Internet of Things Engineering, Jiangxi University of Finance and Economics, Nanchang 330032, China;3.School of Information Technology, Jiangxi University of Finance and Economics, Nanchang 330032, China) Abstract
Objective Images captured in rainy days are blurred due to the influence of raindrops or rain lines, which directly affects the accuracy of outdoor vision tasks, such as image segmentation, object recognition, and object tracking. Image deraining technology aims to recover the lost details in the target scene by detecting and removing the rain streak information in the rain image. According to the imaging principle of the image, the rain streak information in the image can be regarded as a kind of additive noise. Many traditional methods can use some prior information such as the direction, morphological structure and color of rain lines to remove rain noise and obtain a background layer without rain. However, due to the incomplete detection of rain streak information, many methods cannot completely remove rain streaks, resulting in residual rain streaks in the rain removal results. Method Based on the existing problems, we propose a rain removal method based on adaptive morphological filtering and multi-scale convolutional sparse coding. First, according to the diversity of the width and length of the rain streak structure, a set of multi-scale structuring element operators is constructed by using the dilation operation in the morphology to filter out the rain streak information in the image. The size of the filter operator determines the degree of filtering. If the size of the filter operator is too large, more information will be filtered out, resulting in the loss of image texture. On the contrary, if the size of the filter operator is too small, the rain streak information cannot be completely filter out, resulting in the residual rain streak information in the image. Therefore, in order to control the degree of morphological filtering and avoid filtering out too much texture information of the image itself, an adaptive morphological filtering method is proposed. The method controls the size of the filter operator by comparing the wavelet filtering results with the morphological filtering results, so as to obtain low-frequency components with less rain streak information and rich texture information. Next, this method aims to preserve the texture information of the image itself in the low-frequency component as much as possible. Therefore, according considering the inevitable loss of image details in image filtering, the steerable total variation (STV) regularization method is used to detect the texture information of the image itself filtered out in the filtering, and supplement it to the low-frequency components. to obtain good deraining results, detecting accurate rain layer information is the key to deraining methods. Based on the above steps, a high frequency component (i.e., rain layer) containing rain streak information the rain layer can be obtained by subtracting the detected low frequency component from the rain image. Then, to further refine the rain layer, we propose a high-frequency separation method based on multi-scale convolution sparse coding (MS-CSC) to reduce the non-rain streak information in the high-frequency components and obtain a more accurate rain layer. In this process, in order to better detect the rain pattern information, by analyzing the falling direction of rain, two gradient constraints for the horizontal direction x and the vertical direction y are constructed to constrain the solution of the rain layer. The rain grain gradient information mainly exists in the x direction, and the gradient information in the y direction is sparse. Therefore, because the gradient information of the rain streaks in the x direction is relatively rich, this paper constructs the gradient sparse term in the x direction by using the difference between the high frequency component and the reconstructed rain layer to ensure the rain streak reconstruction in the main direction y. Since the rain streak gradient information in the y direction is sparse, the gradient sparse term of the rain layer in the y direction is directly constructed to constrain the rain pattern information in the x direction. By combining the CSC method and the constructed two gradient constraints, a directional gradient-regularized MS-CSC model is constructed to solve for an accurate rain layer. In this model, a dictionary needs to be constructed to implement the encoding of the rain layer. According to the diversity of rain streak structures in image scenes, this paper constructs a multi-scale dictionary to encode rain streak information of different sizes by using dictionaries of different sizes. Finally, the image de-rained result (i.e., the background layer) is obtained by subtracting the rain layer from the rain image. Result To verify the effectiveness of the proposed method, we compared it with some state-of-the-art deraining methods, including some traditional and deep learning-based methods. Our demonstrated results illustrate that the proposed average peak signal-to-noise ratio(PSNR) and structural similarity(SSIM) index is improved by 0.95 and 0.005 2 based on the simulated dataset. Moreover, our simulation method has its priority of qualitative visual effects and quantitative evaluation indexes. Conclusion The integrated MS-CSC and adaptive morphological filtering method can remove rain streaks effectively. The recovery image has its clearer edge texture and details after rain removal. Additionally, our deraining algorithm method has strong generalization capability.
Keywords
image deraining adaptive morphological filter total variation model directional gradient regularization multi-scale convolution sparse coding(MS-CSC)
|