The Subtle Effects of Blood Circulation Can Be Used to Detect Deep Fakes

You probably have etched in your mind the first time you saw a synthetic video of a someone that looked good enough to convince you it was real. For me, that moment came in 2014, after seeing a commercial for Dove Chocolate that resurrected the actress Audrey Hepburn, who died in 1993.

Awe about what image-processing technology could accomplish changed to fear, though, a few years later, after I viewed a video that Jordan Peele and Buzzfeed had produced with the help of AI. The clip depicted Barack Obama saying things he never actually said. That video went viral, helping to alert the world to the danger of faked videos, which have become increasingly easy to create using deep learning.

Dubbed deep-fakes, these videos can be used for various nefarious purposes, perhaps most troublingly for political disinformation. For this reason, Facebook and some other social-media networks prohibit such fake videos on their platforms. But enforcing such prohibitions isn't straightforward.

Facebook, for one, is working hard to develop software that can detect deep fakes. But those efforts will no doubt just motivate the development of software for creating even better fakes that can pass muster with the available detection tools. That cat-and-mouse game will probably continue for the foreseeable future. Still, some recent research promises to give the upper hand to the fake-detecting cats, at least for the time being.

This work, done by two researchers at Binghamton University (Umur Aybars Ciftci and Lijun Yin) and one at Intel (Ilke Demir), was published in IEEE Transactions on Pattern Analysis and Machine Learning this past July. In an article titled, FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals," the authors describe software they created that takes advantage of the fact that real videos of people contain physiological signals that are not visible to the eye.

In particular, video of a person's face contains subtle shifts in color that result from pulses in blood circulation. You might imagine that these changes would be too minute to detect merely from a video, but viewing videos that have been enhanced to exaggerate these color shifts will quickly disabuse you of that notion. This phenomenon forms the basis of a technique called photoplethysmography, or PPG for short, which can be used, for example, to monitor newborns without having to attach anything to a their very sensitive skin.

Deep fakes don't lack such circulation-induced shifts in color, but they don't recreate them with high fidelity. The researchers at SUNY and Intel found that biological signals are not coherently preserved in different synthetic facial parts" and that synthetic content does not contain frames with stable PPG." Translation: Deep fakes can't convincingly mimic how your pulse shows up in your face.

The inconsistencies in PPG signals found in deep fakes provided these researchers with the basis for a deep-learning system of their own, dubbed FakeCatcher, which can categorize videos of a person's face as either real or fake with greater than 90 percent accuracy. And these same three researchers followed this study with another demonstrating that this approach can be applied not only to revealing that a video is fake, but also to show what software was used to create it.

That newer work, posted to the arXiv pre-print server on 26 August, was titled, How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals." In it, the researchers showed they that can distinguish with greater than 90 percent accuracy whether the video was real, or which of four different deep-fake generators (DeepFakes, Face2Face, FaceSwap or NeuralTex) was used to create a bogus video.

Will a newer generation of deep-fake generators someday be able to outwit this physiology-based approach to detection? No doubt, that will eventually happen. But for the moment, knowing that there's a promising new means available to thwart fraudulent videos warms my deep fake-revealing heart.