面向车型识别的夜间车辆图像增强网络RIC-NVNet
摘 要
目的 由于夜间图像具有弱曝光、光照条件分布不均以及低对比度等特点,给基于夜间车辆图像的车型识别带来困难。此外,夜间车辆图像上的车型难以肉眼识别,增加了直接基于夜间车辆图像的标定难度。因此,本文从增强夜间车辆图像特征考虑,提出一种基于反射和照度分量增强的夜间车辆图像增强网络(night-time vehicleimage enhancement network based on reflectance and illumination components,RIC-NVNet),以增强具有区分性的特性,提高车型识别正确率。方法 RIC-NVNet网络结构由3个模块组成,分别为信息提取模块、反射增强模块和照度增强模块。在信息提取模块中,提出将原始车辆图像与其灰度处理图相结合作为网络输入,同时改进了照度分量的约束损失,提升了信息提取网络的分量提取效果;在反射分量增强网络中,提出将颜色恢复损失和结构一致性损失相结合,以增强反射增强网络的颜色复原能力和降噪能力,有效提升反射分量的增强效果;在照度分量增强网络中,提出使用自适应性权重系数矩阵,对夜间车辆图像的不同照度区域进行有区别性的增强。结果 在模拟夜间车辆图像数据集和真实夜间车辆图像数据集上开展实验,从主观评价来看,该网络能够提升图像整体的对比度,同时完成强曝光区域和弱曝光区域的差异性增强。从客观评价分析,经过本文方法增强后,夜间车型的识别率提升了2%,峰值信噪比(peak signal to noiseratio,PSNR)和结构相似性(structural similarity,SSIM)指标均有相应提升。结论 通过主观和客观评价,表明了本文方法在增强夜间车辆图像上的有效性,经过本文方法的增强,能够有效提升夜间车型的识别率,满足智能交通系统的需求。
关键词
RIC-NVNet:night-time vehicle enhancement network for vehicle model recognition
Yu Ye1,2, Chen Weixiao1,2, Chen Fengxin1,2(1.School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230009, China;2.Anhui Province Key Laboratory of Industry Safety and Emergency Technology, Hefei 230009, China) Abstract
Objective The recognition of vehicle model based on night-time vehicle images is challenging for such constraints like weak exposure,irregular distribution of light conditions,and low contrast of night-time images. Specifically, vehicle images taken at night suffer from noise,over-exposure,under-exposure,and additional light source interference, making it difficult for artificial calibration with naked eyes,and recognition with artificial intelligence systems,thus,it directly degrades the performance of intelligent transportation systems. Conventional low-light methods directly brighten the whole image straightforward in terms of histogram equalization,or the correlation of adjacent pixels,which remains being improved on the noise suppression and color distortion. Once the Retinex theory has been proposed,subsequent researches followed the guidance of the theory to decompose the input image into illumination and reflectance components, then enhance the components in a traditional fashion. These methods performed well in the low-light enhancement areas and are limited because of the need for prior knowledge. The emerging deep learning technique-based methods have facilitated low-light image enhancement to a certain extent. Most of them adopted the U-Net and designed a great variety of loss functions to converge the network for better performance. Multiple categories of datasets were proposed and developed to optimize data-driven methods. However,few methods focused on night-time vehicles in the real scenario. Thus,these methods fail to generalize in night-time vehicle enhancement,especially on the aspect of vehicle light interference or underexposure of distinctive vehicle parts. Therefore,considering enhancing the night-time vehicle images,this paper proposes a night-time vehicle image enhancement network based on reflectance and illumination components (RIC-NVNet) to enhance the distinctive features so as to improve both the overall enhancement and the correct rate of vehicle model recognition. Method The RIC-NVNet model consists of an information extraction module,a reflection enhancement module,and an illumination enhancement module. Firstly,the RIC-NVNet utilizes an information extraction module based on the U-Net network structure,using a combination of the night-time vehicle image and its grayscale image as input,to extract the reflection and illumination components of the night-time image. Subsequently,the reflection enhancement module,with a skip connection structure,corrects the color distortion and additional noise problems of the reflection component of the night-time image to obtain an enhanced reflection component. Then,the illumination enhancement module,based on a generative adversarial network structure and an adaptive weight coefficient matrix,generates a day-time illumination component from the illumination component of the night-time image extracted by the information extraction module. Finally, based on the Retinex theory,the enhanced reflection component and the generated daytime illumination component are multiplied to obtain the image after illumination enhancement. To effectively train the RIC-NVNet,we improve the constraint loss of the illumination component to enhance the component extraction effectiveness of the information extraction network. Also,we use color restoration loss,structure consistency loss,and RGB channel loss to constrain the reflection enhancement module to further improve the model’ s performance. In addition,we adopt a generative adversarial loss to constrain the illumination enhancement module and improve its robustness. In summary,the RIC-NVNet is a powerful night-time vehicle image enhancement model that can effectively improve the quality and recognition rate of night-time images. Result On one hand,the performance of RIC-NVNet was evaluated on the simulated night-time vehicle datasets (SNV)and real night-time vehicle datasets(RNV)proposed in this paper. The results showed that using the RIC-NVNet method for low-light image enhancement on these datasets resulted in higher Top1 and Top5 recognition rates obtained by residual neural network-50 (ResNet50)compared to other low-light image enhancement methods. In the SNV dataset,the Top1 and Top5 recognition rates of RIC-NVNet were 82. 68% and 94. 92%,respectively,which were about 2% higher than the lower recognition rates of the zero-reference deep curve estimation (Zero-DCE)method. Additionally,the image quality evaluation indices peak signal to noise ratio(PSNR)and structural similarity(SSIM)were also correspondingly improved compared to other methods. Conclusion The experimental results show that the proposed method can solve the problem of low recognition rates of night-time vehicle images caused by weak exposure and multiple interfering light sources. The method combines an information extraction module,a reflection enhancement module,and an illumination enhancement module,and outperforms other low-light enhancement methods in terms of objective recognition rates,image evaluation indices,and subjective overall image quality of the enhanced night-time vehicle images.
Keywords
vehicle model recognition low light enhancement image decomposition generative adversarial network (GAN) Retinex model
|