Color Changes Due to Blood Circulation Can be Used to Detect Deep Fakes
takyon writes:
The Subtle Effects of Blood Circulation Can Be Used to Detect Deep Fakes
This work, done by two researchers at Binghamton University (Umur Aybars Ciftci and Lijun Yin) and one at Intel (Ilke Demir), was published in IEEE Transactions on Pattern Analysis and Machine Learning this past July. In an article titled, "FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals," [DOI: 10.1109/TPAMI.2020.3009287] [DX] the authors describe software they created that takes advantage of the fact that real videos of people contain physiological signals that are not visible to the eye.
In particular, video of a person's face contains subtle shifts in color that result from pulses in blood circulation. You might imagine that these changes would be too minute to detect merely from a video, but viewing videos that have been enhanced to exaggerate these color shifts will quickly disabuse you of that notion. This phenomenon forms the basis of a technique called photoplethysmography, or PPG for short, which can be used, for example, to monitor newborns without having to attach anything to a their very sensitive skin.
Deep fakes don't lack such circulation-induced shifts in color, but they don't recreate them with high fidelity. The researchers at SUNY and Intel found that "biological signals are not coherently preserved in different synthetic facial parts" and that "synthetic content does not contain frames with stable PPG." Translation: Deep fakes can't convincingly mimic how your pulse shows up in your face.
The inconsistencies in PPG signals found in deep fakes provided these researchers with the basis for a deep-learning system of their own, dubbed FakeCatcher, which can categorize videos of a person's face as either real or fake with greater than 90 percent accuracy. And these same three researchers followed this study with another demonstrating that this approach can be applied not only to revealing that a video is fake, but also to show what software was used to create it.
That newer work, posted to the arXiv pre-print server on 26 August, was titled, "How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals." In it, the researchers showed they that can distinguish with greater than 90 percent accuracy whether the video was real, or which of four different deep-fake generators (DeepFakes, Face2Face, FaceSwap or NeuralTex) was used to create a bogus video.
6 months later: Deep Fakes Get the Blood Pumping.
Related:
Deep Fakes Advance to Only Needing a Single Two Dimensional Photograph
MIT Team Creates Deepfake of President Nixon Reading "Moon Disaster" Apollo 11 Contingency Speech
Microsoft Announces a Deepfake Detector Tool
Read more of this story at SoylentNews.