目的 由于不同伪造类型样本的数据分布差距较大,现有人脸伪造检测方法的准确度不够高,而且泛化性能差。为此,本文引入“图像块归属纯净性”和“残差图估计可靠性”的概念,提出了基于图像块比较和残差图估计的人脸伪造检测方法。方法 除了骨干网络,本文的人脸伪造检测神经网络主要由纯净图像块比较模块和可靠残差图估计模块两部分组成。为了避免在同时包含人脸和背景像素的图像块上提取的混杂特征对于图像块比较的干扰,纯净图像块比较模块中选择只包含人脸像素的纯净人脸图像块和只包含背景像素的纯净背景图像块,通过比较两种图像块纯净特征之间的差异来检测伪造图像,图像块的纯净性保障了特征提取的纯净性,从而提高了特征比较的鲁棒性。考虑到靠近伪造边缘的像素比远离伪造边缘的像素具有较高的残差估计准确度,本文在可靠残差图估计模块中根据像素到伪造边缘的距离设计了一个距离场加权的残差损失来引导网络的训练过程,使网络重点关注输入图像与对应真实图像在伪造边缘附近的差异,对于可靠信息的关注进一步增强了伪造检测的鲁棒性。结果 在FF++(FaceForensics++)数据集上的测试结果显示:与对比算法中性能最好的F2Trans-B相比,本文方法的准确率和AUC(area under the ROC curve)指标分别提高了2.49%和3.31%,在FS(FaceSwap)与F2F(Face2Face)两种伪造数据上的准确率指标分别提高了6.01%和3.99%。在泛化性能方面,与11种已有方法在交叉数据集上的测试结果显示:本文方法与其中性能最好的方法相比,在CDF(Celeb-DF)数据集上的视频AUC指标和图像AUC指标分别提高了1.85%和1.03%。结论 与对比方法相比,由于提高了特征信息的纯净性和可靠性,本文提出的人脸图像伪造检测模型的泛化能力和准确率优于对比方法。

Objective The face recognition technique has become a part of our daily lives in recent years.However,with the rapid development of face forgery techniques based on deep learning,the cost of face forgery has not only been considerably reduced,but unexpected risks to the face recognition technique have also been raised.If someone uses a fake face image to break into a face recognition system,our personal information and property will be compromise and may even be stolen.However,distinguishing whether the face in an image is forged is difficult for the human eyes.Moreover,existing face forgery detection methods exhibit poor generalization performance and are difficult to defend against unknown attack samples due to large data distribution gaps among different forgery samples.Therefore,a reliable and general face forgery detection method is urgently required.In this regard,we introduce the concepts of “patch attribution purity” and “residual estimation reliability”,and propose a novel multitask learning network(PuRe) based on pure image patch comparison(PIPC) and reliable residual map estimation(RRME) to detect face forgery images.Method Apart from the network backbone,our neural network consists of the PIPC module and the RRME module.Both modules are helpful for improving the performance of face forgery detection.On the one hand,if the face in an image is forged,then the features extracted from face and background patches should be inconsistent.The PIPC module compares feature discrepancy between face and background patches to complete the face forgery detection task.Nevertheless,if an image patch contains face and background pixels,then the features extracted from it will have mixed face and background information,disturbing the feature comparison between face and background image patches and resulting in the overfitting of the training dataset.Considering the aforementioned problem,our PIPC module suggests using only pure image patches,which only contain face pixels(pure face image patches) or background pixels(pure background image patches).The purity of image patches guarantees the purity of the extracted features,and thus,the robustness of feature comparison is improved.On the other hand,the residual map estimation task is designed to predict the difference between the input image and the corresponding real image,causing the network backbone to strengthen the generalization of the extracted image features and improving the accuracy of face forgery detection.However,for pixels that are far from the forged edges between the forgery and real regions,the known information used to estimate the residuals will be less,resulting in unreliable residual estimation.Considering the aforementioned problem,a loss function,called the distance field weighted residual loss(DWRLoss),is designed in the RRME module to compel the neural network to give more attention to estimating the residuals near the forged edges between the forgery and real regions.In the face region(i.e.,forgery region),if the pixel is far from the background region,then its loss is assigned with a smaller weight coefficient.Attention to reliable residual information improves the robustness of face forgery detection.Finally,we adopt the multitask learning strategy to train the proposed neural network.Both learning tasks guide the network backbone together to extract effective and generalized features for face forgery detection.Result A large number of experiments are conducted to demonstrate the superiority of our method.Compared with existing superior methods,the test results on the FaceForensics++(FF++) dataset show that the accuracy(ACC) and area under the receiver operating characteristic curve(AUC) of face forgery detection are improved by 2.49%and 3.31%,respectively,by using the proposed method.Moreover,our method improves the ACC of face forgery detection on the FF++ dataset with FaceSwap(FS) and Face2Face(F2F) forgery types by 6.01% and 3.99%,respectively.In terms of the cross-dataset test,compared with 11 existing representative methods,the experimental results show that AUC on the Celeb-DF(CDF) dataset in the video and image levels is increased by 1.85% and 1.03%,respectively,with our method.Conclusion The proposed neural network(i.e.,PuRe) based on the PIPC and RRME modules exhibits excellent generalization ability and performs better than existing methods due to the purity and reliability of the extracted features.