The deepfake avatars who want to sell you everything

This story first appeared in China Report, MIT Technology Review's newsletter about technology developments in China.Sign upto receive it in your inbox every Tuesday.

Writing about China, a question I always get is: What technology is ubiquitous there but hasn't caught on in the West? One of my go-to answers is livestream e-commerce.

If you don't live in China, it's hard to comprehend just how massively popular livestream e-commerce is. Over 500 million Chinese people are watching it regularly; these streams brokered $4.6 trillion in sales last year-meaning more than one-quarter of all purchases made online in China were from livestreams.

It's not surprising that tech companies have been trying to bring that lucrative business outside China. I covered this push as early as in 2020, when AliExpress, Alibaba's overseas arm, was hiring foreign talent to create similar shopping streams in English, Russian, and other languages. But the idea has never been accepted by mainstream viewers elsewhere, even though Chinese companies keep trying.

Most recently, TikTok announced last week that it's officially rolling out its livestream shopping features in the US. (Even this has a sense of deja vu; it's something the company has been saying it's always about to do ... since 2021.)

Nevertheless, it's safe to say that the appetite for livestream shopping still doesn't exist in the US. Juozas Kaziuknas, an e-commerce analyst who founded the firm Marketplace Pulse, tweeted that even fervent fans of TikTok aren't necessarily excited about it, quoting another tweet from last week: tiktok shop has ruined the whole app, it's all ads and reviews instead of silly little videos."

All that said, Chinese livestreamers now have an unprecedented opportunity to break through-with the help of AI.

Today, I published a story about how cheap and convenient AI tools are enabling brands to create countless deepfake streamers on China's e-commerce platforms. The developers require only one minute of video for training a streamer," and charge about $1,000 for AI-generated avatars that can speak and act (almost) like real humans in front of the cameras. They have already been deployed in thousands of shopping livestreams. Read the full story here.

This may help eliminate one of the biggest obstacles to bringing Chinese-style livestream shopping abroad: the lack of foreign talent who understand how livestream e-commerce works, perform naturally in front of the camera, and are willing to talk for hours every day following a largely repetitive routine. In China, there are schools that specifically train aspiring streamers for such work, but the same ecosystem simply doesn't exist anywhere else yet.

Sometimes e-commerce brands take innovative measures to address the talent gap. Chen Dan, CEO of Chinese AI streamer marketing company Quantum Planet, says he saw a Chinese bluetooth headphone brand hire Thai-speaking voice actors to record audio, which then plays over video of just a hand presenting and testing the products, making it seem as if the same person showcasing the product is also speaking Thai.

But with large language models and text-to-speech technologies, these AI streamers can say whatever you want them to say-meaning they can speak other languages too.

Just last week a new AI-translation product was blowing up on social media: LA-based HeyGen launched a tool that translates video into seven different languages, clones the speaker's voice, and syncs the speaker's lips so everything looks natural. The result (including translation to Hindi, the only non-Western language offered now) is surprisingly good!

With tools like this, it's no longer necessary to find local talent for livestreams. Language is actually an advantage of virtual streamers [compared to humans]. Many of our clients are interested in doing cross-border e-commerce in Southeast Asia. The demand is very high," says Huang Wei, the director of virtual influencer livestreaming business at the Chinese AI company Xiaoice.

Xiaoice and Quantum Planet now work together to pitch these AI streamers to Chinese clients. Their virtual streamers can speak 129 languages, including English and a few Southeast Asian languages, like Vietnamese, Thai, and Indonesian.

In March, they used a Thai-speaking AI streamer for the first time to sell furniture for a Chinese company, and sold $2,000 worth of products in an hour. I asked a native Thai speaker to watch a clip and assess the quality of the AI; he told me the intonation was so impressively natural that he almost thought it had to be the result of voice dubbing.

There's also an English version so you can judge for yourself, although I don't think it's on par with the Chinese or Thai versions.

Demo video of an English-speaking livestream influencer generated by Xiaoice.XIAOICE

Obviously the AI won't be able to do everything a human streamer can, especially testing products in real time in response to audience questions, but it suits the companies that are just looking to break into a new market and not spend too much money for the risky venture. As the Chinese publication Huxiu reports, the monthly salary for a local streamer in Indonesia is almost the same as the cost for customizing an AI streamer, and in the long term, it costs much less to reuse the AI than to keep a real person on the payroll. Plus, the result is better than most people expect.

Could this mean that livestream e-commerce will finally get popular outside China? I'd be very cautious and say that it probably won't be the case soon. But I do think AI could help Chinese companies expand globally by overcoming language and cultural barriers. And either way, it's clear the technology of synthetic media is moving ahead at an incredibly fast pace, so it may only be a matter of time before Chinese e-commerce companies can finally capitalize on it.

What do you think of the English version of the AI streamer? Do you think livestream e-commerce will become popular outside China? Let me know your thoughts at zeyi@technologyreview.com.

Catch up with China

1. The Biden administration and TikTok have come back to the negotiating table, after a hard-line stance against the viral social media company has proved unpopular among many Americans. (Washington Post $)

• Two African countries banned TikTok in August, claiming they needed to protect security and morality. Three more countries might follow suit. (Rest of World)

2. Tinder might be disappointing as a dating platform, but in China, young people are using it to find professional connections and job referrals. (Sixth Tone)

3. China's AI sector is overcrowded with newcomers who want to make it big, but the hype is gradually fading away. (Wired $)

4. As the demand for data annotation rises this year, vocational schools in China are working with annotation companies to force students to do the labor-intensive work for subminimum wages. (Rest of World)

5. The US and China are both racing to develop AI tools for espionage-hoping to gain more information on the mindset of their rival's leaders and military capabilities. (New York Times $)

• General Mark Milley, chairman of the Joint Chiefs of Staff, says the bizarre Chinese spy balloon seen earlier this year didn't successfully collect any intelligence data because its sensors were never activated. (CBS News)

6. From his days as a student dissident who escaped Tiananmen in 1989 to his career as one of the best investors betting on Chinese companies, Li Lu's life story is deeply intertwined with the wax and wane of US-China relations over the past three decades. (Financial Times $)

7. Barbie had a team of clearance specialists review every prop for copyright and other concerns. It never expected a cartoonish map to create controversy over China's maritime territory claims. (Wall Street Journal $)

Lost in translation

In the past month, a major Chinese mobile game developer tried twice to set up alternative micropayment systems that circumvent Apple's high commission fees, but it seems to have failed both times. miHoYo is the developer behind the global hit Genshin Impact, which incentivizes players to make micropayments in exchange for characters and items and has become one of the highest-grossing mobile games in the world. That means its players have paid billions of dollars through Apple's App Store-where the US tech giant takes a 30% cut for every transaction.

According to the Chinese gaming publication Core Esports, in August miHoYo tried to set up new payment channels through its community forum app and then a mini-program in the Chinese digital wallet app Alipay. The first app was soon removed from Apple's App Store, while the second was disabled for iPhone users after just 13 days. The failed attempts to reduce revenue sharing with Apple highlight the latter's continued influence in the mobile gaming ecosystem. But Apple is also facing regulatory pressure to open up to third-party payment methods, and major game developers have their own leverage. Maybe the third time will be the charm for miHoYo?

One more thing

If you live in the Chinese province of Guangdong, I'd advise you to stay indoors. After heavy rainfall and flooding across southern China, 71 crocodiles escaped from a farm where they were raised for their meat and leather. The good news is that 69 of them have been caught. The bad news is ... well, there are still two at large.