Towards Untrusted Social Video Verification to Combat Deepfakes via Face Geometry Consistency
Abstract
Deepfakes can spread misinformation, defamation, and propaganda by faking videos of public speakers. We assume that future deepfakes will be visually indistinguishable from real video, and will also fool current deepfake detection methods. As such, we posit a social verification system that instead validates the truth of an event via a set of videos. To confirm which, if any, videos are being faked at any point in time, we check for consistent facial geometry across videos. We demonstrate that by comparing mouth movement across views using a combination of PCA and hierarchical clustering, we can detect a deepfake with subtle mouth manipulations out of a set of six videos at high accuracy. Using our new multi-view dataset of 25 speakers, we show that our performance gracefully decays as we increase the number of identically faked videos from different input views.
Cite
Text
Tursman et al. "Towards Untrusted Social Video Verification to Combat Deepfakes via Face Geometry Consistency." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020. doi:10.1109/CVPRW50498.2020.00335Markdown
[Tursman et al. "Towards Untrusted Social Video Verification to Combat Deepfakes via Face Geometry Consistency." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.](https://mlanthology.org/cvprw/2020/tursman2020cvprw-untrusted/) doi:10.1109/CVPRW50498.2020.00335BibTeX
@inproceedings{tursman2020cvprw-untrusted,
title = {{Towards Untrusted Social Video Verification to Combat Deepfakes via Face Geometry Consistency}},
author = {Tursman, Eleanor and George, Marilyn and Kamara, Seny and Tompkin, James},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
year = {2020},
pages = {2784-2793},
doi = {10.1109/CVPRW50498.2020.00335},
url = {https://mlanthology.org/cvprw/2020/tursman2020cvprw-untrusted/}
}