'Deepfake face swap' app ZAO is making people afraid of the future
The 'deepfake-style face swap app' ZAO has climbed to the top of Android and iPhone download charts in recent weeks. As its popularity grew, so have privacy concerns on Chinese social media, and now, beyond.
Here's how it works:
In case you haven't heard, #ZAO is a Chinese app which completely blew up since Friday. Best application of 'Deepfake'-style AI facial replacement I've ever seen.
Here's an example of me as DiCaprio (generated in under 8 secs from that one photo in the thumbnail)
pic.twitter.com/1RpnJJ3wgT
- Allan Xia (@AllanXia) September 1, 2019
The sudden wide adoption of ZAO is an "intriguing development in a country where mass surveillance and facial recognition technology are prevalent," writes Jake Newby at radiichina.com.
"Some social media platforms, including WeChat, have now started blocking ZAO videos," Newby writes in an update to his story on Monday. "WeChat has done this before with popular rival short video apps."
The app - developed by Momo, the same company behind popular Chinese dating app Tantan - became an overnight sensation after it began circulating on Friday evening. Hashtags related to the app quickly became some of the hottest on microblogging site Weibo, while the app rocketed up the iOS download charts. Chinese social media feeds quickly became filled with ZAO-produced videos from friends and contacts for many users.
The premise of the app is pretty simple: take a selfie and put yourself into your favorite movie or soap opera (chosen from a pre-selected list of clips). Cue users giving themselves starring roles in Leonardo DiCaprio's filmography or uninvited guest appearances on Game of Thrones.
Yet the initial buzz around the app quickly gave way to headlines about privacy concerns as reporters from several major tech publications in China actually bothered to read the 6,000 character user agreement required to download ZAO. One clause in particular is causing consternation as it appears to give the app's developers the global rights to use any imagery created on the app for free. Once a user has opted in, there doesn't seem to be the right to revoke the agreement.
This story went viral pretty quickly, and ZAO "responded to the above complaints" by amending the user agreement, writes Newby in another update to the Radii piece:
The new agreement states that content generated on the app will now no longer be used for other purposes without the user's prior consent. It also says that if a user deletes content on ZAO, this will be wiped from ZAO's databases too, and injects a clause stating that users bear responsibility for having the authorization to use portraits in the app (thereby making the user culpable if someone's photo is used without their permission).
[via techmeme.com]
"Alipay, the country's biggest mobile payment service ... has been forced to reassure users that ZAO's deepfakes can not be used to bypass their security checks."
Viral Deepfake App ZAO Sparks Mass Downloads, Memes and Major Concerns https://t.co/8BmkFRtZzI
- Tom Barraclough (@Tom_Bcgh) September 2, 2019
If there is anything you need to do that depends on video being treated as evidence, your remaining time is limited. https://t.co/J8ELGyXaD0
- Eliezer Yudkowsky (@ESYudkowsky) September 2, 2019
The fact that this technology is now easily available to everyone is terrifying. In addition to all the misinformation already on the internet, get ready for fake soundbites and deep fake interviews. https://t.co/XoY2JUEBiO
- Hala Gorani (@HalaGorani) September 2, 2019
"Launched recently, Zao is currently topping the free download chart on China's iOS store. Its popularity has also pushed another face-swap app, Yanji, to fifth place on the list" https://t.co/eWSi8PhWO3 via @technology
- Fabio Chiusi (@fabiochiusi) September 2, 2019
How long before this deep fake tech finds it's most profitable application is in porn? https://t.co/gEbP7izUGF
- Rollo Tomassi (@RationalMale) September 2, 2019
I wonder if these #ZAO-type experiments can well be beneficial. When #DeepFake technology becomes so easily available to everyone, it might goad us all to be a lot more skeptical of "I readily believe my eyes and ears" stance--an anachronism in this age of perceptual #AI.. https://t.co/bgNVcY5aOT
- Subbarao Kambhampati (@rao2z) September 1, 2019
It's telling that these days people instantly jump to the worst applications of new tech.
This was once science fiction level tech in movies like Running Man and now it's available in a free app anyone can download. https://t.co/0WqAxLcTjH
- Dare Obasanjo (@Carnage4Life) September 2, 2019
We are exactly 30 seconds away from this tech being used to end a marriage, wreck a career, put someone in prison, or start a war. https://t.co/f8eeujBKFQ
- Myke Cole (@MykeCole) September 1, 2019