非规律运动伪迹干扰鲁棒的人脸视频心率检测
摘 要
目的 基于远程光电容积脉搏波描记法(remote photoplethysmograph,rPPG)的非接触人脸视频心率检测广泛应用于移动健康监护领域,由于其携带的生理参数信息幅值微弱,容易受到运动伪迹干扰。据此,提出了一种结合非负矩阵分解(nonnegative matrix factorization,NMF)和独立向量分析(independent vector analysis,IVA)的非规律运动伪迹去除的视频心率检测方法,记为 NMF-IVA。方法 首先,将面部感兴趣区域(region of interest,RoI)分为多个子区域(sub RoIs,SRoIs),利用平均光照强度、光照强度变化、信噪比这 3 个指标筛选出 3 个最优质的 SRoIs,并获取每个 SRoI 的绿色通道时间序列。其次,将 3 个绿色通道时间序列去趋势、带通滤波后送入 NMF-IVA 进行盲源分离。然后,对分离后的源信号进行功率谱密度分析,并且将峰值信噪比最高且主频落在心率感兴趣范围内的源信号确定为血容量脉冲(blood volume pulse,BVP)信号。最后,将 BVP 信号的主频确定为所测量心率的主频,从而计算出心率值。结果 实验在 2 个公开数据集 UBFC-RPPG 和 UBFC-PHYS,及 1 个真实场景自采数据集上与最相关的 7 种典型的 rPPG 方法进行比较,在 UBFC-RPPG 数据集上,相比于性能第 2 的单通道滤波(single channel filtering,SCF)方法,均方根误差提升了 1. 39 bpm(beat per minute)、平均绝对误差提升了 1. 25 bpm、皮尔逊相关系数提升了 0. 02;在UBFC-PHYS 数据集上的 T2 情况下,其性能提升最为显著,相比于性能第 2 的独立向量分析(IVA)方法,均方根误差提升了 16. 42 bpm、平均绝对误差提升了 9. 91 bpm、皮尔逊相关系数提升了 0. 64;在自采数据集上,除了低于深度学习方法性能之外,所提 NMF-IVA 方法在传统方法中取得了最好的结果。结论 所提 NMF-IVA 方法对规律信号提取具有敏感性,即便是在头部存在剧烈非规律运动情况下,相比于传统方法亦能取得最优结果,该结果能够媲美基于深度学习的方法。
关键词
Facial video-based heart rate measurement against irregular motion artifacts
Cheng Juan1,2, Yin Chenchu1, Song Rencheng1, Fu Jing1, Liu Yu1(1.Department of Biomedical Engineering, Hefei University of Technology, Hefei 230009, China;2.Anhui Province Key Laboratory of Measuring Theory and Precision Instrument, Hefei University of Technology, Hefei 230009, China) Abstract
Objective Heart rate (HR) is one of the most important physiological parameters that can reflect the physical and mental status of individuals. Various methods have been developed to estimate HR values using contact and noncontact sensors. The advantage of noncontact methods is that they provide a more comfortable and unobtrusive way to estimate HR and avoid discomfort or skin allergy caused by conventional contact methods. The pulse-induced subtle color variations of facial skins can be measured from consumer-level cameras. Thus, camera-based non-contact HR detection technology, also called remote photoplethysmograph (rPPG), has been widely used in the fields of mobile health monitoring, driving safety, and emotion awareness. The principle of camera-based rPPG measurement is similar to that of traditional PPG measurement, that is, the pulsatile blood propagating in cardiovascular systems changes blood volumes in microvascular tissue beds beneath skins with each heartbeat, thus producing a fluctuation. However, such technology is susceptible to motion artifacts due to weak amplitudes of the physiological parameter information it carries. For instance, subjects’ heads may move involuntarily during interviews, presentations, and other socially stressful situations, thus degrading rPPG-based HR detection performance. Accordingly, this paper proposes a novel motion-robust rPPG method that combines nonnegative matrix factorization (NMF) and independent vector analysis (IVA), termed as NMF-IVA, to remove irregular motion artifacts.Method First, the whole facial region of interest (RoI) is divided into several sub RoIs (SRoIs), among which three optimal SRoIs are selected based on three indicators: average light intensity, light intensity variation of a certain SRoI, and signal-to-noise ratio (SNR) of the green-channel signal derived from the SRoI. Afterwards, three green-channel time series are derived from the corresponding three optimal SRoIs. Second, the three channels of time series are detrended, bandpass filtered, and then sent to the proposed NMF-IVA as input. After the NMF-IVA operation, three source signals are extracted and then processed by power spectral density analysis. The one with the highest peak SNR and the corresponding dominant frequency falling within the interested HR range will be identified as the blood volume pulse (BVP) signal, whose dominant frequency is identified as that of the estimated HR.Result We compare the proposed NMF-IVA method with seven typical rPPG methods on two publicly available datasets(UBFC-RPPG and UBFC-PHYS) as well as one in-house dataset. On the UBFC-RPPG dataset, compared with the second-best performance of the single channel filtering (SCF) method, the proposed NMF-IVA achieves better performance, with an improved root mean square error (RMSE) of HR measurement by 1.39 beat per minute (bpm), an improved mean absolute error (MAE) by 1.25 bpm, and a higher Pearson’s correlation coefficient (PCC) by 0.02. Although both the MAE and the RMSE achieved by the proposed NMF-IVA method are lower than those of deep learning-based methods, the PCC of the NMF-IVA is comparable to that of deep learning-based ones, which demonstrates the effectiveness of the proposed NMF-IVA method. As for the UBFC-PHYS dataset when compared with traditional rPPG methods, during the T1 condition, the performance of the proposed NMF-IVA method is better than that of the second-best SCF method, with an improved RMSE by 6.45 bpm, an improved MAE by 2.53 bpm, and a higher PCC by 0.18. When compared with deep learning-based ones, the proposed NMF-IVA method achieves the second-best performance. The performance improvement of the proposed NMF-IVA is most noticeable during the T2 condition on the UBFC-PHYS dataset. Specifically, when compared with the second-best performance of IVA, the above three metrices are improved by 16.42 bpm, 9.91 bpm, and 0.64, respectively. As for the UBFC-PHYS dataset, when during the T3 condition, the best performance is still achieved by the proposed NMF-IVA method. When compared with the second-best performance of the independent component analysis method, the corresponding three metrices are improved by 8.54 bpm, 6.14 bpm, and 0.37, respectively. The performance of the proposed NMF-IVA method can be comparable to that of deep learning-based ones both in T2 and T3 conditions. As for the in-house dataset, the proposed NMF-IVA method achieves better performance compared with the traditional methods, except for deep learning-based methods.Conclusion The proposed NMF-IVA method achieves the best results on all the three datasets when compared with traditional rPPG methods, and the performance improvement is most noticeable during irregular motion artifact conditions involving head motions with large amplitudes. However, the performance of the proposed NMF-IVA method is slightly poorer than that of deep learning-based methods possibly because deep learning technology has excellent abilities in learning and extracting effective features. However, sufficient training samples and generalization should be considered when adopting deep learning-based methods. In addition, before the high-quality BVP source is derived, upsampling is employed, which leads to a relatively large time consumption. In the future, the HR estimation performance and the upsampling rate should be traded off. The proposed NMF-IVA method has advantages in extracting regular signals. Thus, our study can provide a new solution for promoting the practical application ability of rPPG technology.
Keywords
remote photoplethysmograph (rPPG) non-contact heart rate measurement blind source separation(BSS) nonnegative matrix factorization (NMF) independent vector analysis (IVA)
|