The Chinese government’s problematic quest to judge online comments

China Report is MIT Technology Review's newsletter about technology developments in China. Sign up to receive it in your inbox every Tuesday.

This morning, I published an explainer on China's social credit system. The government released a draft law on November 14 that will eventually serve as the top-level guidance on how the country builds the system.

That's right, the national Chinese social credit system is not even fully built yet-but you'd be forgiven for thinking otherwise. Right now, it only really exists in fractured pieces. Still, there are a lot of Western misperceptions about it-particularly that there is an all-knowing algorithm that scores individual people's daily behaviors. This doesn't exist in reality, and in my story I explain what the system actually is and how we should understand social credits.

The real social credit system focuses on promoting trustworthiness in business, consumption, education, and ... almost every other aspect of life. That sounds pretty good in theory, but in today's newsletter, I want to dig into how this is all more complicated than it seems.

One particular example of the social credit system's implications-specifically, how it can affect social media and freedom of speech-reveals how the noble-sounding goal of building trust can be problematic in practice.

The key question here is: Who is judging whether a social media comment or user is trustworthy? This is an incredibly difficult question that platforms around the world, including Twitter and Facebook, are still struggling to answer.

In China, though, the government seems confident that it can be the ultimate arbiter. So, as Princeton University researcher Shazeda Ahmed tells me, it's necessary to ask: What does being trustworthy or honest mean in the eyes of the Chinese government?

In 2019, a draft regulation was introduced by China's Cyberspace Administration to build a social credit system in the internet sector. It rules that government agencies can deem an individual or a company seriously untrustworthy entities" for fabricating, publishing, or transmitting information contrary to social mores, commercial morality, or honesty and credit."

While some information can be fact-checked and proved false, things like social mores" and honesty" are often too vague to be defined objectively, so it goes back to the government to identify what makes a moral value.

Well, we have already seen how this can go terribly wrong. In the early months of covid-19, the local government in Wuhan punished eight individuals for posting rumors" about the virus that allegedly undermined public trust. Li Wenliang, the whistleblower doctor who told friends in a group chat that a SARS-like virus was spreading, was summoned by the police for posting lies online." Later, his death from covid ignited a national wave of fury against the government's control of information.

This is just one incident, but as the idea of building social creditworthiness increasingly seeps into other regulations, it reveals the risks of standardizing a practice wherein the government makes moral judgments for its people.

Just last week, China's Cyberspace Administration finalized a regulation entirely dedicated to online comments," which I covered when it was first proposed in June. The regulation's main purpose is to place social media interactions, including those in newer forms like livestreams, under the same strict controls China has always had for other online content.

These rules aren't really part of the broader social credit system, but I still found some familiar language in the document. It asks social media platforms to carry out credit assessments of users' conduct in commenting on posts" and conduct credit appraisals of public account producer-operators' management of post comments."

The idea is that if an influencer or a user posts things that are not trustworthy, that should be reflected in the person's credit assessment. And the results of the credit assessment will determine the scope of services and functionality" people are offered on certain platforms.

It's not the only specific example of the Chinese government using importance of creditworthiness" or trust" to justify more rules. This was seen when the government decided to establish a blacklist of celebrities who promote bad" morals, crack down on social media bots and spam, and designate responsibilities to administrators of private group chats.

This is all to say that the ongoing development of China's social credit system is often in sync with the development of more authoritarian policies. As China turns its focus increasingly to people's social and cultural lives, further regulating the content of entertainment, education, and speech, those rules will also become subject to credit enforcement," legal scholar Jeremy Daum wrote in 2021.

Nevertheless, before you go, I do want to caution against the tendency to exaggerate perceived risks, which has happened repeatedly when people have discussed the social credit system.

The good news is that so far, the intersection of social credit and the control of online speech has been very limited. The 2019 draft regulation to build a social credit system for the internet sector has still not become law. And a lot of the talk about establishing credit appraisal systems for social media, like the one requested by the latest regulation on online comments, looks more like wishful thinking than practical guidance at this point. Some social platforms do operate their own credit scores"-Weibo has one for every user, and Douyin has one for shopping influencers-but these are more side features that few in China would say are top of mind.

Today, instead of worrying about the theoretical risk of how a social credit system could be used to stifle freedom of speech, it's more important to put our time and resources into identifying what censorship mechanisms are already in place and running, like those social platforms routinely use to screen posts and ban accounts for discussing politically sensitive events (including the massive ban ahead of the 20th Party Congress). These are larger and more urgent threats to Chinese internet users than a social credit system that even the government hasn't figured out yet.

What other concerns do you have about China's social credit system? I'd love to know. Write me at zeyi@technologyreview.com

Catch up with China

1. Taiwan's parliament passed its own version of the US CHIPS Act, which aims to retain Taiwan's lead in semiconductor manufacturing technology by giving out large corporate tax breaks. (Reuters $)

2. As Meta, Twitter, and Amazon conduct massive layoffs, TikTok is approaching some of their laid-off software engineers and is planning to double the staff at its office in Mountain View, California. (The Information $)

3. TikTok's content moderation subcontractor in Colombia is being investigated by the country's Ministry of Labor for traumatic working conditions and low pay. (Time)

4. Many senior FTX employees have left their free accommodations in the Bahamas and returned to Hong Kong, where the crypto exchange had a headquarters until 2021. (Semafor)

5. Wall Street investors are getting excited about Chinese stocks again, driven by recent covid policy changes in China and the Biden-Xi meeting. (Bloomberg $)

• Silicon Valley may be less confident. Tim Draper, an early Tesla investor, decided to pull out from his investments in China and turn to startups in Taiwan instead. (Wall Street Journal $)

6. Chinese doctors say the country's medical system is not prepared for the inevitable surge in covid cases when Beijing decides to further relax its zero-covid policies. (Financial Times $)

7. China's 20- and 30-year-olds no longer want to work in manufacturing factories like their parents. And automation technologies can't (yet) fill this labor gap. (Reuters $)

Lost in translation

Almost a year after China's top livestream influencer Viya was suddenly censored for tax evasion, she has reinvented herself as the head of a sprawling supply chain business and investment empire, Chinese publication Renwu reported.

Once known for being able to sell over a billion dollars' worth of merchandise in one livestream, Viya (whose real name is Huang Wei) hit rock bottom in December 2021, when she was fined nearly $200 million by the Chinese government. All Chinese social media platforms subsequently banned her accounts. But Viya had already started turning her online influence toward building an e-commerce supply chain ecosystem long before her fall. While she never appeared in front of the public again, her business empire has survived. Two manufacturing startups that she invested in went public recently, and she may have made more cash than when she was still appearing in front of the cameras.

One more thing

Um, that's awkward. Ahead of a rugby competition between Hong Kong and South Korea held in a city near Seoul on November 13, the organizer mistakenly played Glory to Hong Kong," an unofficial anthem from the 2019 pro-democracy protests, instead of the Chinese national anthem. And it soon emerged that in at least two separate rugby competitions this year, the correct anthem was played, but the TV graphics still referred to it as Glory to Hong Kong." Instead of taking it like a champ, the Hong Kong government has launched a police investigation into the latest incident, and a particularly pro-Beijing legislator, Junius Ho, even called for the Hong Kong rugby team to be disbanded for not reacting while the wrong song was played.